SEARCH · Search NASA

Results for “Automated Machine Vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Automating Testing of DUNE Electronics via a Finite State Machine

The Deep Underground Neutrino Experiment (DUNE) is a flagship international collaboration designed to study neutrinos tiny, nearly massless particles that may hold answers to fundamental questions about the Universe. Fermilab s Robotic Test Stand (RTS) plays a critical role in ensuring the quality of approximately 50,000 Application-Specific Integrated Circuit (ASIC) chips that will be used in DUNE s massive liquid argon detectors. These electronics will be inside the cryostat; therefore, they will need to have a high yield of working chips and low noise. To improve the automation and reliability of the RTS, this project focused on designing and implementing a Python-based finite state machine (FSM) to manage chip handling workflows. The FSM was developed as a modular software framework to coordinate robotic arm movements, manage chip tray positions, and monitor system states during testing. Key features include robust error handling routines, a pause/resume system for safe mid-cycle interruptions, and a simulation mode for iterative testing without hardware dependencies. The system was designed to prepare for seamless integration with RTS hardware components such as the robotic arm and vision system. This integration will streamline collaboration and enable efficient deployment of updates across the six total institutions performing testing. The outcomes of this internship contribute to Fermilab s mission to advance high-energy physics and support the DOE s national goals by directly improving the testing of equipment to be used in DUNE. The project also provided valuable experience in software design and contributing to the success of DUNE.

Kang, Caleb [William Rainey Harper Coll.]↗

DOE OSTI2025

Automating Testing of DUNE Electronics via a Finite State Machine

The Deep Underground Neutrino Experiment (DUNE) is a flagship international collaboration designed to study neutrinos—tiny, nearly massless particles that may hold answers to fundamental questions about the Universe. Fermilab’s Robotic Test Stand (RTS) plays a critical role in ensuring the quality of approximately 50,000 Application-Specific Integrated Circuit (ASIC) chips that will be used in DUNE’s massive liquid argon detectors. These electronics will be inside the cryostat; therefore, they will need to have a high yield of working chips and low noise. To improve the automation and reliability of the RTS, this project focused on designing and implementing a Python-based finite state machine (FSM) to manage chip handling workflows. The FSM was developed as a modular software framework to coordinate robotic arm movements, manage chip tray positions, and monitor system states during testing. Key features include robust error handling routines, a pause/resume system for safe mid-cycle interruptions, and a simulation mode for iterative testing without hardware dependencies. The system was designed to prepare for seamless integration with RTS hardware components such as the robotic arm and vision system. This integration will streamline collaboration and enable efficient deployment of updates across the six institutions performing testing. The outcomes of this internship contribute to Fermilab’s mission to advance high-energy physics and support the DOE’s national goals by directly improving the testing of equipment to be used in DUNE. The project also provided valuable experience in software design and contributing to the success of DUNE.

Kang, Caleb [Fermilab]↗

DOE OSTI2025

AI foundation models for experimental fusion tasks

Artificial Intelligence (AI) foundation models, while successful in various domains of language, speech, and vision, have not been adopted in production for fusion energy experiments. This brief paper presents how AI foundation models can be used for fusion energy diagnostics, enabling, for example, visual automated logbooks to provide greater insights into chains of plasma events in a discharge, in time for between-shot analysis.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2026

Illuminating the Material World: Autonomous Microscopy to Understand Order, Disorder, and Everything In Between

Artificial intelligence (AI) holds immense promise for revolutionizing microscopy, yet its widespread adoption has been hindered by challenges ranging from user inexperience to limited model transferability and difficulties in operationalizing machine learning. This presentation showcases our approach to developing practical autonomy for materials discovery, aiming to accelerate the integration of AI into everyday microscopy workflows. As shown in Fig. 1, I will focus on three key areas: understanding order-disorder transitions, quantifying point defects, and achieving truly device-scale microscopy. First, I will demonstrate the power of multi-modal knowledge graphs for integrating diverse microscopy data. By combining imaging, spectroscopy, and diffraction data, these graphs provide a holistic view of material behavior, capturing the intricate relationships between different modalities [1,2]. I will present a case study on how these models illuminate the structural and chemical changes associated with irradiation in oxide thin films, revealing critical insights for designing materials for extreme environments like spaceflight and nuclear energy. Specifically, I will show how multi-modal analysis clarifies the evolution of order-disorder transitions under irradiation, a key factor influencing material performance in these applications. Next, I will address the challenge of quantifying point defects in 2D materials. We demonstrate the application of computer vision and transfer learning to accurately identify and classify various defect types, such as vacancies and substitutional atoms, and to quantify their concentrations. This information is crucial for understanding and tailoring the properties of 2D materials for applications in electronics, optoelectronics, and catalysis. For example, I will show how our models can characterize the topological distribution of point defects in MXene transition metal carbides, providing valuable insights for optimizing their performance in energy storage and separation science. Finally, I will discuss our progress toward autonomous device-scale microscopy [3,4]. We are fundamentally redesigning electron microscopes around the principles of machine reasoning, enabling automation beyond basic tasks like sample navigation and data acquisition to include sophisticated experimental design. This approach paves the way for truly reproducible and massively scaled analysis campaigns. I will emphasize the importance of autonomous microscopy platforms for high-throughput materials discovery and characterization, facilitating the rapid screening of materials for a broad range of applications and accelerating the development of next-generation technologies.

36 MATERIALS SCIENCE↗

DOE OSTI2019

Estimation of full-field, full-order experimental modal model of cable vibration from digital video measurements with physics-guided unsupervised machine learning and computer vision

Cables are critical components for a variety of structures such as stay cables and suspenders of cable-stayed bridges and suspension bridges. When in operational service, they are vulnerable to cumulative fatigue damage induced by dynamic loads (e.g., the cyclic vehicle loads and wind excitation). To accurately analyze and predict their dynamics behaviors and performance that could be spatially local and temporal transient, it is essential to perform high-resolution vibration measurements, from which their dynamics properties are identified and, subsequently, a high spatial resolution, full-modal-order dynamics model of cable vibration can be established. In this study, we develop a physics-guided, unsupervised machine learning-based video processing approach that can blindly and efficiently extract the full-field (as many points as the pixel number of the video frame) modal parameters of cable vibration using only the video of an operating (output-only) cable. In particular, by incorporating the physics of cable vibration (taut string model), a novel automated modal motion filtering method is proposed to enable autonomous identification of full-order (as many modes as possible) dynamic parameters, including those weakly excited modes that used to be challenging to identify in operational modal analysis. Therefore, a full-field, full-order modal model of cable vibration is established by the proposed method. Furthermore, this new approach provides a low-cost and noncontact technique to estimate the cable tension using only the video of the vibrating cable where the fundamental frequency is automatically and efficiently estimated to compute the cable tension according to the taut string equation. Laboratory experiments on a bench-scale cable are conducted to validate the developed approach.

42 ENGINEERING↗

NASA NTRS1991

The Space Station Freedom Flight Telerobotic Servicer: the design and evolution of a dexterous space robot

The Flight Telerobotic Servicer (FTS) Project at the Goddard Space Flight Center is developing an advanced telerobotic system to assist in and reduce crew extravehicular activity (EVA) for Space Station) Freedom (SSF). The FTS will provide a telerobotic capability to the Freedom Station in the early assembly phases of the program and will be employed for assembly, maintenance, and inspection applications throughout the lifetime of the space station. Appropriately configured elements of the FTS will also be employed for robotic manipulation in remote satellite servicing applications and possibly the Lunar/Mars Program. In mid-1989, the FTS entered the flight system design and implementation phase (Phase C/D) of development with the signing of the FTS prime contract with Martin Marietta Astronautics Group in Denver, Colorado. The basic FTS design is now established and can be reported on in some detail. This paper will describe the FTS flight system design and the rationale for the specific design approaches and component selections. The current state of space technology and the nature of the FTS task dictate that the FTS be designed with sophisticated teleoperation capabilities for its initial primary operating mode. However, there are technologies, such as advanced computer vision and autonomous planning techniques currently in research and advanced development phases which would greatly enhance the FTS capabilities to perform autonomously in less structured work environments. Therefore, a specific requirement on the initial FTS design is that it has the capability to evolve as new technology becomes available. This paper will describe the FTS design approach for evolution to more autonomous capabilities. Some specific task applications of the FTS and partial automation approaches of these tasks will also be discussed in this paper.

Task Performance and Analysis↗

DOE OSTI2019

A Streamlined and Standardized Benthic Habitat Mapping Approach for Marine and Hydrokinetic Site Environmental Assessments

We have combined two seafloor mapping techniques to efficiently map benthic habitat conditions. Multibeam echosounder (MBES) surveys provide detailed bathymetry and backscatter mosaics. Sediment profile and plan view imaging (SPI/PV) rapidly provides dense ground truth data on seabed physical and biological features. A key innovation of this project has been the development of a computer-assisted image processing system that standardizes the SPI data generation process. A unique combination of machine- and deep-learning techniques for grain-size classification, sediment–water interface delineation, and object detection (e.g., infauna and feeding voids) is described. An automated SPI image analysis system provides a standardized and cost-effective approach for image processing. Combining MBES with SPI/PV camera technology provides a first tier approach to rapidly documenting seabed physical and biological conditions in high resolution at MHK sites. When coupled with computer automated data extraction from the imagery, this approach provides a standardized and cost-effective method to establish and monitor benthic habitat conditions over time.

Benthic Habitat Maping, Sediment-Profile Imaging, ↗

NASA NTRS2003

The H-Metaphor as a Guideline for Vehicle Automation and Interaction

Good design is not free of form. It does not necessarily happen through a mere sampling of technologies packaged together, through pure analysis, or just by following procedures. Good design begins with inspiration and a vision, a mental image of the end product, which can sometimes be described with a design metaphor. A successful example from the 20th century is the desktop metaphor, which took a real desktop as an orientation for the manipulation of electronic documents on a computer. Initially defined by Xerox, then refined by Apple and others, it could be found on almost every computer by the turn of the 20th century. This paper sketches a specific metaphor for the emerging field of highly automated vehicles, their interactions with human users and with other vehicles. In the introduction, general questions on vehicle automation are raised and related to the physical control of conventional vehicles and to the automation of some late 20th century vehicles. After some words on design metaphors, the H-Metaphor is introduced. More details of the metaphor's source are described and their application to human-machine interaction, automation and management of intelligent vehicles sketched. Finally, risks and opportunities to apply the metaphor to technical applications are discussed.

Flemisch, Frank O.↗

NASA NTRS2010

Feature Extraction and Selection Strategies for Automated Target Recognition

Several feature extraction and selection methods for an existing automatic target recognition (ATR) system using JPLs Grayscale Optical Correlator (GOC) and Optimal Trade-Off Maximum Average Correlation Height (OT-MACH) filter were tested using MATLAB. The ATR system is composed of three stages: a cursory region of-interest (ROI) search using the GOC and OT-MACH filter, a feature extraction and selection stage, and a final classification stage. Feature extraction and selection concerns transforming potential target data into more useful forms as well as selecting important subsets of that data which may aide in detection and classification. The strategies tested were built around two popular extraction methods: Principal Component Analysis (PCA) and Independent Component Analysis (ICA). Performance was measured based on the classification accuracy and free-response receiver operating characteristic (FROC) output of a support vector machine(SVM) and a neural net (NN) classifier.

computer vision↗

DOE OSTI2020

Software-Defined Network for End-to-end Networked Science at the Exascale

Domain science applications and workflow processes are currently forced to view the network as an opaque infrastructure into which they inject data and hope that it emerges at the destination with an acceptable Quality of Experience. There is little ability for applications to interact with the network to exchange information, negotiate performance parameters, discover expected performance metrics, or receive status/troubleshooting information in real time. The work we presen here is motivated by a vision for a new smart network and smart application ecosystem that will provide a more deterministic and interactive environment for domain science workflows. The Software-Defined Network for End-to-end Networked Science at Exascale (SENSE) system includes a model-based architecture, implementation, and deployment which enables automated end-to-end network service instantiation across administrative domains. An intent based interface allows applications to express their high-level service requirements, an intelligent orchestrator and resource control systems allow for custom tailoring of scalability and real-time responsiveness based on individual application and infrastructure operator requirements. This allows the science applications to manage the network as a first-class schedulable resource as is the current practice for instruments, compute, and storage systems. Deployment and experiments on production networks and testbeds have validated SENSE functions and performance. Emulation based testing verified the scalability needed to support research and education infrastructures. Key contributions of this work include an architecture definition, reference implementation, and deployment. This provides the basis for further innovation of smart network services to accelerate scientific discovery in the era of big data, cloud computing, machine learning and artificial intelligence.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Software-defined network for end-to-end networked science at the exascale

Domain science applications and workflow processes are currently forced to view the network as an opaque infrastructure into which they inject data and hope that it emerges at the destination with an acceptable Quality of Experience. There is little ability for applications to interact with the network to exchange information, negotiate performance parameters, discover expected performance metrics, or receive status/troubleshooting information in real time. The work presented here is motivated by a vision for a new smart network and smart application ecosystem that will provide a more deterministic and interactive environment for domain science workflows. The Software-Defined Network for End-to-end Networked Science at Exascale (SENSE) system includes a model-based architecture, implementation, and deployment which enables automated end- to-end network service instantiation across administrative domains. An intent based interface allows applications to express their high-level service requirements, an intelligent orchestrator and resource control systems allow for custom tailoring of scalability and real-time responsiveness based on individual application and infrastructure operator requirements. This allows the science applications to manage the network as a first-class schedulable resource as is the current practice for instruments, compute, and storage systems. Deployment and experiments on production networks and testbeds have validated SENSE functions and performance. Emulation based testing verified the scalability needed to support research and education infrastructures. Key contributions of this work include an architecture definition, reference implementation, and deployment. This provides the basis for further innovation of smart network services to accelerate scientific discovery in the era of big data, cloud computing, machine learning and artificial intelligence.

47 OTHER INSTRUMENTATION↗

DOE OSTI2020

Artificial Intelligence and Computer Vision for Cost-Effective Benthic Habitat Characterizations

Machine learning applied to computer vision and pattern recognition is a type of artificial intelligence that has advanced rapidly in the last 10 to 15 years, spurred forward by breakthroughs in deep convolutional neural networks. These state-of-the-art methods are poised to become widely used in environmental monitoring applications as a result of the increasing abundance of data available from different imaging platforms (e.g., fixed-point cameras, drone surveys, high-resolution satellite data) that can be analyzed to observe, model, and understand environmental conditions. Computer vision and pattern recognition (CVPR) tools advance our ability to use imagery and camera-based tools in cost-effective ways for environmental monitoring. Although these techniques offer great potential, some challenges remain, such as the need for large sets of labeled images for model training and validation and optimized hardware and software to ensure that the models can be trained effectively and in a reasonable amount of time. Here, we have overcome many of these rate-limiting challenges by using a diverse image library built across multiple projects coupled with staff expertise and onsite computing resources. We are working toward a fully automated SPI processing system and also are moving to develop CVPR analytical tools for other imaging platforms and data sets.

02 PETROLEUM↗

DOE OSTI2022

A Combined Computer Vision and Deep Learning Approach for Rapid Drone-Based Optical Characterization of Parabolic Troughs

Optical accuracy is a primary driver of parabolic trough concentrating solar power (CSP) plant performance, but can be damaged by wind loads, gravity, error during installation, and regular plant operation. Collecting and analyzing optical measurements over an entire operating parabolic trough plant is difficult, given the large scale of typical installations. Distant Observer, a software tool developed at the National Renewable Energy Laboratory, uses images of the absorber tube reflected in the collector mirror to measure both surface slope in the parabolic mirror and offset of the absorber tube from the ideal focal point. This technology has been adapted for fast data collection using low-cost commercial drones, but until recently still required substantial human labor to process large amounts of data. A new method leveraging advanced deep learning and computer vision tools can drastically reduce the time required to process images. This new method addresses the primary analysis bottleneck, identifying featureless, reflective mirror corner points to a high degree of accuracy. Recent work has shown promising results using computer vision methods. The combined deep learning and computer vision approach presented here proved highly effective and has the potential to further automate data collection and analysis, making the tool more robust. The method presented in this paper automatically identified 74.3% of mirror corners within 2 pixels of their manually marked counterparts and 91.9% within 3 pixels. This level of accuracy is sufficient for practical Distant Observer analysis within a target uncertainty. A commercial drone collected video of over 100 parabolic trough modules at an operating CSP plant to demonstrate the deep learning and computer vision method's usefulness in processing large amounts of data. These troughs were successfully analyzed using Distant Observer, paired with the new deep learning and computer vision algorithm, and can provide plant operators and trough designers with valuable insight about plant performance, operating strategies, and plant-wide optical error trends.

computer vision↗

DOE OSTI2025

Exocortex Network for AI-Augmented Human-Led Scientific Expedition

AI advances in science can be viewed along two main directions with a fluid boundary: enhancing efficiency through automation and smart tools to accelerate tasks that humans can already perform; and enabling exploration into uncharted territories and potentially toward AGI. These advances manifest in the AI cognitive core through the development and explainability of foundation models; in the physical embodiment of instruments and facilities; and in the integrated agency of AI workflows exemplified by the science exocortex. To address the role of humans in this evolving landscape, in this Perspective, we suggest a third direction: the development of personalized agents that form human-centered networks, supporting both efficiency and exploration while ensuring that AI remains aligned with human vision.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Collaborative Robots vs. Master/Slaves - 19522

This paper will discuss the feasibility, application, and economy of applying collaborative robots (co-bots) in lieu of traditional through-the-wall tele-operated manipulator systems, traditionally used within hazardous (radioactive) environments. Both fixed facility and mobile applications will be discussed. Both single- and dual-armed co-bot systems will be employed. The six-axis, plus gripper, single-armed co-bots will provide for machine vision enhanced pick-n-place applications within shielded rooms and also include force-reflecting (haptic) teleoperations controls for non-routine materials-handling operations. The dual-armed system will be mounted to a motorized stand with motorized adjustment both vertically and rotationally (around its base). To complement these co-bots, both two-dimensional and three-dimensional vision technologies will be implemented. The vision technologies, in these applications, are used in lieu of traditional radiation shielding windows. two-dimensional video systems will provide for large area viewing, incorporating pan, tilt, and zoom capabilities. Three-dimensional video systems will be mounted to the dual-armed robot stand and provide viewing, including depth-of-perception, to the operator, performing material handling and other tasks, from an operator workstation located outside the hazardous environment. The unique workstation used with the dual-armed systems will also be described. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

NASA NTRS2011

Lunabotics Mining Competition: Inspiration Through Accomplishment

NASA's Lunabotics Mining Competition is designed to promote the development of interest in space activities and STEM (Science, Technology, Engineering, and Mathematics) fields. The competition uses excavation, a necessary first step towards extracting resources from the regolith and building bases on the moon. The unique physical properties of lunar regolith and the reduced 1/6th gravity, vacuum environment make excavation a difficult technical challenge. Advances in lunar regolith mining have the potential to significantly contribute to our nation's space vision and NASA space exploration operations. The competition is conducted annually by NASA at the Kennedy Space Center Visitor Complex. The teams that can use telerobotic or autonomous operation to excavate a lunar regolith geotechnical simulant, herein after referred to as Black Point-1 (or BP-1) and score the most points (calculated as an average of two separate 10-minute timed competition attempts) will eam points towards the Joe Kosmo Award for Excellence and the scores will reflect ranking in the on-site mining category of the competition. The minimum excavation requirement is 10.0 kg during each competition attempt and the robotic excavator, referred to as the "Lunabot", must meet all specifications. This paper will review the achievements of the Lunabotics Mining Competition in 2010 and 2011, and present the new rules for 2012. By providing a framework for robotic design and fabrication, which culminates in a live competition event, university students have been able to produce sophisticated lunabots which are tele-operated. Multi-disciplinary teams are encouraged and the extreme sense of accomplishment provides a unique source of inspiration to the participating students, which has been shown to translate into increased interest in STEM careers. Our industrial sponsors (Caterpillar, Newmont Mining, Harris, Honeybee Robotics) have all stated that there is a strong need for skills in the workforce related to robotics and automated machines. In 2010, 22 United States (US) universities competed, and in May 2011 the competition was opened to international participation, with 46 Universities attending. There were 12 international teams and 34 US teams. This combined total directly inspired an estimated 544 university students. More students and the public were engaged via internet broadcasting and social networking media. This is expected to be of value for actual future space missions, as knowledge is gained from testing many innovative prototypes in simulated lunar regolith. More information is available at www.nasa.gov/lunabotics/.

Mueller, Robert P.↗

DOE OSTI2020

Machine Vision-based Robot Manipulators for Nuclear Applications - 20370

Decommissioning and dismantling of nuclear facilities are major challenges facing the nuclear industry. Robot manipulators with capabilities of restoring nuclear structures and significantly prolonging nuclear power generation in addition to their applications in decommissioning and dismantling (D and D) would immensely ease the above-mentioned challenge. Ability to remotely sense/monitor and analyze dangerous and hazardous environments such as nuclear reactors as well as performing reparatory tasks in such environments using robot manipulators require additional information from vision sensors. Structural sample collection and inspection as well as restoration require object and position information. Computer vision is used to obtain precise 3D position information. The current work uses the Sawyer robot with the integrated Cognex camera. The research presents an approach to facilitate and improve operations in a nuclear reactor using robotic vision control. The vision control system is modeled with Augmented Image Space-based visual servoing approach. Results showing accurate robot control. Object recognition is used to recognize the image and calculate it poses. To transform coordinates of the object's pose from the camera frame to the robotic frame transformation matrices are employed. Future work will employ a 3D camera (with depth information) using Denso robot. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2022

Automated processing of environmental transmission electron microscopy images for quantification of thin film dewetting and carbon nanotube nucleation dynamics

We report scalable production of carbon nanotubes (CNTs) requires catalysts and reaction conditions that provide high nucleation efficiency. In situ characterization methods such as environmental transmission electron microscopy (ETEM) can reveal fundamental mechanisms of synthesis, but to date have primarily provided qualitative observations on small sample sizes. Here, quantitative analysis is performed using high-resolution, high-rate video capture of ETEM experimentation coupled with automated image processing, involving computer vision algorithms and convolutional neural networks. By this approach, we detect distinct nanoparticle formation from an alumina-supported iron thin film and subsequent CNT nucleation from the nanoparticles. The statistical summary of particles in each video shows that, compared to a H 2 -only atmosphere, pretreatment of the catalyst with carbon added to the H 2 atmosphere results in a smaller average particle diameter, a 2-fold increase in particle density (to 5300 particles/μm 2 ), a 3-fold increase in CNT nucleation efficiency (to 92%), and more than a 5-fold increase in CNT density (to 4800/μm 2 ). Addition of carbon during exposure to H 2 is also more effective than NH 3 at dewetting the catalyst film and increasing the CNT nucleation efficiency, in spite of NH 3 being a stronger reducing agent for iron. Insights from this study are applicable to improving CNT yield and productivity in both batch-style and continuous processes.

25 ENERGY STORAGE↗