SEARCH · Search NASA

Results for “Vision based Navigation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

11 records

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Autonomous Aerial Power Plant Inspection in GPS-denied Environments

Inspection of coal-fired power plants is frequently dangerous, includes difficult places to reach, and can turn expensive due to the downtimes and cost of inspection crew. Robotic systems have shown capabilities to address some of these issues, but most of the current robotic inspection technology in power plants is designed for specific components. Conversely, recent advances in machine vision have empowered aerial platforms for long-range, remotely-controlled, GPS-based inspections of industrial plants. This capability has led to wide spread utilization of aerial robots (commonly termed Drones, UVS or UAS) platforms for inspection in less challenging environments where both collision avoidance, and GPS reception are not significant issues. The challenge in adapting airborne technology for power plant inspection lies in internal structures and the complex network of piping, and distribution systems, which impose significant risks for collision and can hinder the reception and transmission of GPS signals. The current state of the art in aerial inspection technology within the energy sector is controlled via radio control, and utilizes GPS-based navigation, for inspection of large-scale plants such as offshore platforms and wind turbine parks. Nevertheless, close-range and autonomous inspection in the GPS-denied environments of power plants has not yet been achieved, as it requires precise guidance and navigation with real-time situational awareness and obstacle avoidance capabilities. This endeavor introduced the use of rotary wing flying robots, due to their station keeping and vertical take-off capabilities for power plant components inspection. To enable close quarter inspection two methods were used. One method uses the 3D CAD (Three-dimensional Computer-Aided Design) model of the asset to inspect to generate the UAV’s inspection path. To acquire, analyze and process the 3D model, first, the STL file is produced to obtain surface points and vectors normal to the surface. Later, by introducing other variables such as wall offset and a controlled trajectory between each outline and each subsequent layer, the flight path is generated. The proposed framework will generate a path that will pass as close as desired from the surface and navigate in intricate environments. A second method, use advanced manufacturing techniques such as CNC (Computer Numerical Control) and additive manufacturing. Once the inspection flight path is obtained, vision-based navigation systems are employed to have the UAV autonomously tracking the provided trajectory. Finally, Artificial Intelligence-enabled developments are in charge of detecting cracks and corrosion in structural components of power plants. The proposed methods are validated in simulations, laboratory and industrial setups, where it is shown that the developed systems acting together enable close-quarter autonomous aerial inspection and mapping in power plant assets. The system can be further improved by adding more sensors to navigate in different GPS-denied environments, with non-homogeneous lighting conditions, dust and in general situations where vision-based systems may fail.

01 COAL, LIGNITE, AND PEAT↗

DOE OSTI2024

A Vision–based Robust $\mathcal{H}$ ∞ Gain Scheduling Longitudinal and Lateral Following Controller for Autonomous Vehicles on Urban Curved Roads

Implementing advanced driver assistance systems (ADAS) in congested and intricate urban traffic scenarios poses significant challenges. To address the frequent stop–and–go motions exhibited by autonomous vehicles (AVs) navigating urban roads with changes in curvature, we propose a vision–based robust $\mathcal{H}$ ∞ adaptive cruise control system (ACC) for longitudinal control, plus a lane keeping assist system (LKAS) for lateral control. For the vision-based ACC, a weighted probability objective function for the vehicle following behavior is formulated. We incorporate $\mathcal{H}$ ∞ performance and gain scheduling techniques to mitigate the impact of uncertainty in visual sensor measurements. Furthermore, the optimal time headway is scheduled based on the velocity to ensure traffic flow efficiency and safety during the vehicle following process. For the LKAS, we introduce a road curvature estimation method that integrates lane and vehicle dynamics information to obtain the lateral and heading offsets. Next, the design criterion of the observer–based robust gain scheduling lateral motion controller is established by linear matrix inequality (LMI). Here, a series of experiments conducted within a camera–in–loop platform validate the proposed method.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2025

Mechanically flexible mid-wave infrared imagers using black phosphorus ink films

The mid-wave infrared (MWIR) spectral range (λ = 3–8 μm) enables important sensing and imaging applications, including non-invasive bioimaging, night vision, and autonomous navigation. Commercial MWIR photodetectors are limited to rigid imagers based on heteroepitaxial materials. There is an emerging need for mechanically flexible MWIR imagers to broaden their functionality and practicality. Recently, photodetectors using van der Waals (vdW) black phosphorus (BP) flakes have demonstrated highly sensitive room-temperature photodetection. Additionally, vdW materials are solution-processable, facilitating scalable processing and flexible device fabrication. In this work, we present flexible MWIR imagers consisting of photodiodes fabricated on thin plastic substrates using BP ink films. We demonstrate mechanically robust responsivity up to 2.5-mm bending radii and after 5000 bending cycles. Leveraging this flexibility, we achieve full-azimuthal imaging, detecting directional light sources with precision. These results establish a scalable approach for large-area, conformable MWIR imaging and pave the way for integration with flexible electronics.

Wijaya, Theodorus Jonathan↗

DOE OSTI2022

Intelligent Process Visualization through Nuclear Operation Process Modeling, Reasoning, and Object Detection from Field Videos (Final Report)

This report is a deliverable for the “Final Report” task of DOE NEET Project 19-16790, "Context-Aware Safety Information Display for Nuclear Field Workers." This project's overall goal is to test the hypothesis that integrating computer vision and process reasoning methods will enable proactive visualization of the safe operation and maintenance processes of Nuclear Power Plants (NPP) for field workers. Augmented Reality (AR) glasses adopting such proactive safety information visualization techniques can significantly increase personnel safety and reduce the NPP’s operating costs. The current practice of monitoring NPPs requires workers to switch between digital models, data, and physical workspaces in identifying relevant but potentially occluded objects and in assessing the risks of operation and maintenance processes. On the other hand, frequently changed field conditions require field workers to report to supervisors for real-time guidance. Such guidance is essential to ensure that changing conditions will not invalidate or endanger the work order and other ongoing processes that may jeopardize NPP operations. Additionally, incorrect recognition of equipment objects can result in communication errors and safety problems. AR techniques can assist engineers in viewing the physical workspaces with objects labeled with detailed operation procedures and safety reminders during field operations. The project team developed an “Intelligent Context-Aware Safety Information Display” (ICAD) for supporting Nuclear Power Plant (NPP) field workers in achieving safe and efficient execution of a series of operational tasks in uncertain and changing workspaces of an NPP. Before designing the ICAD-AR prototype, the project team synthesized NPP operational knowledge models through literature review studies, surveys, interviews with domain experts, and knowledge modeling. The project team conducted an extensive study of the operational procedures of various NPPs, and digital technologies that can support the safe and efficient execution of those procedures in different NPP operational contexts. This literature review helped the project team conduct surveys and interviews with nuclear engineers and field workers to identify three categories of information. The NPP knowledge modeling efforts reveal that the three categories of information identified have different levels of importance in a typical procedure of carrying out a series of tasks to achieve a specific NPP operation goal (e.g., shutdown, mode changes). These three categories of information include 1) Workspace dynamics – the changing spatial arrangements of workspaces, tools, protection equipment, and supporting materials, 2) Workflow prognostics – the dynamic dependencies between different parts of an NPP that functionally support and influence each other in terms of safety and efficiency, and 3) Hazards – objects and spaces that contain hazardous materials or physical conditions that can pose risks to workers or mechanical systems. The project team has profiled the importance levels of these categories of information into a knowledge model. This knowledge model specifies what types of information are more critical for a given task in a given workspace so that computers can automatically identify critical objects and sensors in a scene for delivering context-ware safety information to field workers through AR devices. Significant research development of this project results in technical research outcomes and a prototyping system that illustrates the technical feasibility of establishing an ICAD-AR system supporting the proactive safety information display for nuclear field workers. This final report summarizes the project team’s technological achievements in the past three years. Overall, the project team completed the development and integration of five techniques into a prototype ICAD Augmented Reality (ICAD-AR) system and demonstrated the developed system’s real-time execution in a mechanical room. The project team completed the analysis of using this prototype in other types of workspaces based on 3D image data and digital design models collected from two additional workspaces (a water treatment plant and a flow loop training facility). The integrated techniques include 1) Natural Language Processing (NLP) algorithms supporting the generation and updates of nuclear fieldwork process models based on text analysis of work packages and operation manuals; 2) sensor log analysis for predicting control actions in given sensor reading contexts; 3) computer vision algorithms for automatic localization and navigation of workers; 4) object detection algorithms for identifying task-related objects and correlated sensors for safety checking; 5) AR technique as a platform for supporting the integration. The testing results of these five techniques have shown that 1) the sensor log analysis model can predict the next control action with an accuracy of 0.883; 2) the trained natural language processing model can extract more than 80% of the critical information from paper-based procedures (PBPs); 3) the navigation algorithm with the integration of Visual Inertial Odometry (VIO) and Non-Recursive Bayesian Filter methods make operator’s trajectory estimation resilient to drift error; 4) the computer vision algorithm can detect task-specific and safety-critical objects with an average accuracy of 95.3%. The project team used work procedures collected from a flow loop training facility and two datasets collected from two mechanical rooms simulating the workspaces of NPPs to demonstrate the technical capabilities of the developed ICAD-AR prototype. The demonstration validated the technical feasibility of establishing the ICAD-AR system for nuclear field workers and identified the challenges in 1) automatic text analysis of work packages; 2) use of limited samples of sensor logs for predicting the proper timings of control actions; 3) reliably tracking workers and their task progress in mechanical rooms with many similar objects.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

Methods and Comparisons Between Computer Vision and Radar Based Vehicle Location

Measuring vehicle locations relative to a driver's vehicle is a critical component in the analysis of driving data from both postanalysis (such as in naturalistic driving studies) or in autonomous vehicle navigation. In this work we describe a method to estimate vehicle positions from a forward-looking video camera using intrinsic camera calibration, estimates of extrinsic parameters, and a convolutional neural network trained to detect and locate vehicles in video data. We compare the measurements we achieve with this method with ground truth and with radar data available from a naturalistic driving study. We identify regions where video is preferred, where radar is preferred, and explore trade-offs between the two methods in regions where the preference is more ambiguous. We describe applications of these measurements for transportation analysis.

Aykac, Deniz↗

DOE OSTI2024

Accelerating Computational Materials Discovery with Machine Learning and Cloud High-Performance Computing: from Large-Scale Screening to Experimental Validation

High-throughput computational materials discovery has promised significant acceleration of the design and discovery of new materials for many years. Despite a surge in interest and activity, the constraints imposed by large-scale computational resources present a significant bottleneck. Furthermore, examples of large-scale computational discovery carried through experimental validation remain scarce, especially for materials with product applicability. In this paper, we demonstrate how this vision became reality by first combining state-of-the-art artificial intelligence (AI) models and traditional physics-based models on cloud high performance computing (HPC) resources to quickly navigate through more than 32 million candidates and predict around half a million potentially stable materials. Focusing on solid-state electrolytes for battery applications, our discovery pipeline further identified 18 promising candidates with new compositions and rediscovered a decade’s worth of collective knowledge in the field as a byproduct. By employing around one thousand virtual machines in the cloud, this process took less than 80 hours. We then synthesized and experimentally characterized the structures and conductivities of our top candidates, the Na x Li 3-x YCl 6 (0.5 ≤ x ≤ 2.5) series, demonstrating the potential of these compounds to serve as solid electrolytes. Additional candidate materials are currently under experimental investigation that could offer more examples of the computational discovery of new phases of Li- and Na-conducting solid electrolytes. We believe this unprecedented approach of synergistically integrating AI models and cloud HPC not only accelerates materials discovery but also showcases the potency of AI-guided experimentation in unlocking transformative scientific breakthroughs with real-world applications.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Microchannel-based Membrane-less Extraction of Li from Unconventional Lithium Sources & the Separation of REE

This final report provides an overview of the Project's entire duration, covering July 1, 2021 to December 31, 2023. It primarily focuses on the achievements, technological developments, and unique challenges the team faced while working on separating and extracting Lithium from produced waters. The project's primary aim was to create an integrated, high-throughput, membrane-less, and modular microfluidic platform that could extract Lithium from unconventional sources. We have successfully met all goals and milestones envisioned in the SOPO document. The most critical primary milestones, including the Go-No-Go milestone (refer to the Gantt chart in the Appendices), were successfully accomplished. We demonstrated phase separation (>90%) and extraction (>85%) performance in the MPSE using synthetic, and representative produced water composition feed at 50 ml/min total flow through MPSE 36. We have also performed a parametric study of the MPSE operations, beyond the scope of SOPO, exploring operating conditions of current and broader interest. The extended investigation of operational parameters is concurrent with our efforts to seek further development of the MPSE technology beyond the scope of the Project. Along these lines of development, we have made efforts to be responsive to DOE calls for technological developments of other types of resources (beyond PW) for the recovery of Critical Materials and higher TRL development (beyond TRL 4). During the work on this Project, we developed and implemented three innovative technical approaches that emerged from our efforts to successfully meet the Project milestones. The innovative & original technical approaches developed and implemented in this Project are now the contributions to process engineering that could be clearly credited to the Project. First, Convergent Design Approach is a comprehensive feedforward & feedback loop of four design phases: i) design for functionality, ii) design for manufacturing, iii) design for sustainability, and iv) design for market. Next was Process Intensification. A major aim of this Project was to create an innovative phase separation & extraction microscale-based technology for Li separation – thus the words microchannel-based in the Project title. A microscale-based technology is intrinsically in the center of the Process Intensification domain as defined by its unique principles. Therefore, Process Intensification was implicitly envisioned in the Project’s SOPO. Lastly, Time Scale Analysis is a novel tool for discovering the needs and directions of Process Intensification implementations in any process technology. This Project is fully credited for developing and implementing the three novel technical approaches mentioned above. These are general contributions to process engineering that emerged from this Project. Beyond the original SOPO scope, the OSU-U.Pitt research group utilized a Convergent Design methodology, integrating first-principles mathematical modeling with experimental validation on the Minimum Development Vehicle. By creating these Digital Twins, the team rapidly assessed manufacturing iterations to support TEA analysis. This framework further enabled the development of advanced Surface Modification Techniques, where hydrophobic and oleophobic coating strategies were optimized via Digital Twin tools and validated through rigorous 100-hour longevity testing. TEA Analysis: The closing efforts of this Project were focused on the TEA analysis. TEA analysis had two primary functions: i) enabling critical assessments of design variations withing 10 the Concurrent Design Approach, thus enabling evolution of the MPSE design to reach faster- better-cheaper alternatives; and ii) to create a bridge between the accomplishments of this Project and future projects of higher TRL, beyond TRL 6 level. It is important to note that the TEA model created in the Project stirred the technological solutions for the recovery of critical materials toward a vision of a very profitable modular plant that has unique zero-waste water discharge signature. More importantly, thanks to our experimental performance data and conservative assumptions, the TEA model predicts minimal technological and investment risks. Low cost of a modular unit of a nominal capacity of [1000 tons of Li 2 CO 3 /year] positions the MPSE based technology within the reach of community investors, thus offering a paradigm shift in the development of critical technologies. The project successfully navigated two primary challenges: solvent selection and manufacturing adaptation. Restricted by the SOPO to existing literature for lithium recovery, the team identified a critical need for a "material excellence program" to develop next-generation solvents, eventually concluding with a preliminary investigation into promising Ionic Liquids (ILs). Simultaneously, COVID-19 supply chain disruptions forced a pivot from traditional manufacturing to advanced additive methods at ATAMI-OSU. By transitioning from stainless steel to 3D-printed polymer substrates, the team achieved a transformative three-order-of- magnitude reduction in manufacturing costs and compressed prototyping timelines from several months to just two days. The MPSE technology offers significant energy, environmental, and economic advantages by overcoming the traditional bottlenecks of phase-separation hardware and contactor size. Unlike conventional mixer-settlers or membrane-based systems, MPSE operates without moving parts or fouling-prone membranes, achieving robust performance even with challenging, viscous, or particulate-heavy feeds. Key performance metrics include an energy intensity reduction of 5–50x (3–40 kJ/m 3 ) compared to incumbent technologies and a dramatic reduction of processing time to under 60 seconds, which drastically reduces the physical plant footprint. These technical efficiencies translate into superior economic outcomes; for a 100 t/year Li 2 CO 3 facility, implementing MPSE is projected to nearly halve contactor CAPEX (from $\$$6.08M to $\$$3.01M) and significantly increase the project's Net Present Value (NPV), derisking new investment and enabling distributed critical-mineral processing configurations. The commercialization of MPSE technology is being spearheaded by Vigsur Dynamics Inc., which has adopted a structured, parallel approach to technical and business development since its formation in January 2026. Following extensive customer discovery and engagement with the Oregon State University accelerator, Vigsur Dynamics is working to establish a business model that transitions from pilot demonstrations to modular hardware sales, ultimately aiming for a "build-own-operate" service strategy. Current technical milestones—including 100 hours of continuous operation, superior energy efficiency, and successful 6-unit modular scale-up— provide a foundation for this transition. Backed by ongoing IP licensing and a growing network of industrial and venture advisors, the company is actively de-risking the platform to replace conventional mixer-settler systems in the critical minerals market.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

Vegetation Detection Using Deep Learning and Conventional Methods

Land cover classification with the focus on chlorophyll-rich vegetation detection plays an important role in urban growth monitoring and planning, autonomous navigation, drone mapping, biodiversity conservation, etc. Conventional approaches usually apply the normalized difference vegetation index (NDVI) for vegetation detection. In this paper, we investigate the performance of deep learning and conventional methods for vegetation detection. Two deep learning methods, DeepLabV3+ and our customized convolutional neural network (CNN) were evaluated with respect to their detection performance when training and testing datasets originated from different geographical sites with different image resolutions. A novel object-based vegetation detection approach, which utilizes NDVI, computer vision, and machine learning (ML) techniques, is also proposed. The vegetation detection methods were applied to high-resolution airborne color images which consist of RGB and near-infrared (NIR) bands. RGB color images alone were also used with the two deep learning methods to examine their detection performances without the NIR band. The detection performances of the deep learning methods with respect to the object-based detection approach are discussed and sample images from the datasets are used for demonstrations.

58 GEOSCIENCES↗

DOE OSTI2020

Constructing Self-Labeled Materials Imaging Datasets from Open Access Scientific Journals with EXSCLAIM!

Due to recent improvements in image resolution and acquisition speeds, materials microscopy is experiencing an explosion in imaging data. Yet, despite the volume of images generated, the overall accessibility landscape is highly fragmented, as researchers who do release images to the public, often only do so as snapshots of their larger private dataset in context of scientific journal publications. The effort to automatically consolidate images and descriptive information from web-based platforms has garnered broad attention from the computer vision, language technologies, and chemistry/materials informatics communities. However, these methods are problematic for scientific figures because over 30% of figures are compound in nature, and it is the individual images themselves, paired with relevant context, that are necessary for construction a proper labeled dataset. To this end, we outline in this paper the design of a software pipeline for the automatic EXtraction, Separation, and Caption-based natural Language Annotation of IMages from scientific figures (EXSCLAIM!). Successful consolidation of materials imaging across literature sources will enhance navigation and searchability of materials microscopy images for both novice and experienced researchers, as well as establish the framework necessary for users to search by images, text, or some combination of both.

36 MATERIALS SCIENCE↗

DOE OSTI2024

Real World Use Case Evaluation of Radar Retro-reflectors for Autonomous Vehicle Lane Detection Applications

Lane detection plays a critical role in autonomous vehicles for safe and reliable navigation. Lane detection is traditionally accomplished using a camera sensor and computer vision processing. The downside of this traditional technique is that it can be computationally intensive when high quality images at a fast frame rate are used and has reliability issues from occlusion such as, glare, shadows, active road construction, and more. This study addresses these issues by exploring alternative methods for lane detection in specific scenarios caused from road construction-induced lane shift and sun glare. Specifically, a U-Net, a convolutional network used for image segmentation, camera-based lane detection method is compared with a radar-based approach using a new type of sensor previously unused in the autonomous vehicle space: radar retro-reflectors. This evaluation is performed using ground truth data, obtained by measuring the lane positions and transforming them into pixel coordinates. The performance of each method is assessed using the statistical R2 score, indicating the correlation between the detected lane lines and the ground truth. The results show that the U-Net camera-based method exhibits limitations in accurately detecting and aligning the lane lines, particularly in challenging scenarios. However, the radar-based lane detection method demonstrates a strong correlation with the ground truth which implies that the use of this sensor may improve current reliability issues from conventional camera lane detection approach. Furthermore, the study highlights the limitations of the U-Net model for camera lane detection, especially in scenarios with sun glare. This study shows that infrastructure-based radar retro-reflectors can improve autonomous vehicle lane detection reliability. The integration of different sensor modalities and the development of advanced computer vision algorithms are crucial for improving the accuracy, reliability, and energy efficiency of lane detection systems. Addressing these challenges contributes to the advancement of autonomous vehicles and the realization of safer and more efficient transportation systems.

Brown, Nicolas E.↗

DOE OSTI2023

Building Environmental Justice and Equity into the Development of Critical Mineral Industries

The global transition toward low-carbon energy not only means an increase in demand for clean electricity and renewable resources, but also an increase in demand for the critical minerals (CMs) and rare earth elements (REEs) that these technologies rely upon. Indeed, low-carbon energy technologies, such as those used for wind turbines and electric vehicle motors, require significantly more lithium, nickel, cobalt, manganese, graphite, and other CMs than fossil-based energy technologies. The growing demand for new forms of low- carbon energy will necessitate a proportional scale-up of CM and REE extraction and processing. Presently, the majority of extraction and processing activities for CMs and REEs are concentrated in very few countries, primarily in China and the Global South. CM and REE supply chain activities conducted in or controlled by these countries are commonly associated with widespread and well- documented human rights abuses and environmental degradation. As a result of concerns related to supply chain security, worker welfare, and the overall need for additional sources of CMs and REEs, governments worldwide have begun to explore policy pathways for the development of new supply chains. The development of new supply chains for CMs and REEs re s a multifold opportunity to accelerate the global deployment of low-carbon energy fleets, reduce global carbon emissions, build political resilience—and also to enhance policy objectives of environmental justice and equity in the clean energy transition. Fulfilling these policy objectives requires energy producers to navigate a maze of global supply chain policies designed to shape and accelerate the growth of new markets. In the United States, for instance, President Biden recently implemented fiscal and trade policies that incentivize both domestic and global reliance on U.S.-produced CMs and REEs, leveraging key relationships in Asia and Europe to ensure the accelerated buildout of the U.S. supply chain. The Biden Administration’s framework also emphasizes that new supply chains for CMs, REEs, and other low-carbon energy pathways must generate benefits for marginalized and disadvantaged communities, build energy equity, and contribute to the Biden Administration’s vision of environmental justice. present CM and REE supply chains include several stages, including mining, processing, transport, utilization, and disposal, each of which involves different environmental justice considerations and potential injustices. This study explores how technical innovation, paired with responsible community engagement and empowerment, can help inform the development of energy equity and environmental justice at all stages of new supply chains. The study highlights these opportunities on a broad scale. We also lend a specific focus to research at the University of Wyoming School of Energy Resources that aims to guide the development of CM and REE industries in communities with high economic dependence on coal and other fossil fuel industries, identifying pathways to grow a U.S. domestic supply chain by producing CMs from coal, coal by-products, and coal waste streams. These production pathways represent a potential new supply for CMs across the U.S. and elsewhere, thereby enhancing national security and accelerating the widespread deployment of low-carbon energy technologies, while also generating alternative applications for remaining coal reserves and aiding in a just transition for rural energy-producing communities.

Gerace, Selena↗