Search NASA⌕ Search

SEARCH · Search NASA

Results for “COMPUTER VISION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Improvements in Reflected Target Non-intrusive Assessment (ReTNA)

Accurate optical surfaces are a primary driver of concentrated solar power plant performance. Errors in pointing and tracking mirrors, the canting of individual mirror facets, and the surface slope of the mirror itself can be caused by errors during assembly, transportation, wind loading, gravity, and many other sources. The tools that exist to measure these error sources today largely rely on fringe deflectometry (SOFAST, QDec, others), or photogrammetry with targets attached to the mirror surface. In 2022, NREL presented a new indoor optical measurement system to measure fully assembled heliostats during development and on the assembly line for quality assurance purposes, which has undergone significant development in the last year, including the installation of a new, larger laboratory setup. This system, called ReTNA (Reflected Target Non-intrusive Assessment), uses a printed pattern as a deflectometry target. Using printed pattern features as deflectometry targets instead of fringes allows the software to measure the surface slope in two dimensions, eliminating the need to for changing, projected targets. In the last year, NREL's ReTNA tool has been adapted to use a tall, thin target, that only spans one dimension of the mirror, and a camera moving on an automatic track system. We will present these new developments, including the new setup, target patterns, software methodology for fast, automatic optical analysis, and important lessons learned throughout the process. ReTNA results will be compared to established tools, to establish the uncertainty of this new method. ReTNA leverages recent improvements in computer vision and photogrammetry to remove as much careful calibration from the setup process as possible, making the tool low cost, simple, and portable. While it will not achieve the resolution of a fringe deflectometry system, its advantages make it well suited for CSP evaluation, and allow it to fill a different industry need.

concentrating solar power↗
DOE OSTI2024

Power System Waveform Datasets for Machine Learning

The desire for increased visibility across the electricity grid will necessarily increase the deployment of sensing and measurement devices and associated data management needs to unprecedented levels. For the existing sensing and measurement infrastructure, there remains a great amount of “value” yet to be extracted through advanced data management and analytics. Availability of more data will not, by itself, lead to changes in grid visibility, security, and resiliency. To create the predictive and prescriptive environment required to enable new markets and transactions for customer revenue and a reliable grid, the data must be collected, organized, evaluated, and analyzed using sophisticated algorithms to provide actionable information allowing operators and customers to reliably manage an increasingly complex grid. Progress in artificial intelligence (AI) has been largely driven by large, publicly available datasets that can be used to train AI algorithms such as MNIST, a database of handwritten images of digits, and ImageNet, an image database of everyday objects. These types of publicly available databases of real-world training datasets have been largely credited for advancement of image processing, computer vision, and deep learning algorithms that these use cases deploy. However, in the power systems industry to date, there are few databases with proper event labeling, and data access to a publicly available collection of power system event waveforms that will allow users to interact with grid signature data. Publicly available datasets of power system event waveforms, such as the DOE/EPRI dataset, often lack critical metadata or contain limited examples of each event type, and data formats vary widely across these datasets.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Identifying Neutrino Final States and Energies in MicroBooNE with New Deep-Learning Based LArTPC Reconstruction Frameworks

MicroBooNE, a Liquid Argon Time Projection Chamber (LArTPC) located in the $\nu_{\mu}$-dominated Booster Neutrino Beam at Fermilab, has been studying $\nu_{e}$ charged-current (CC) interaction rates to shed light on the MiniBooNE low energy excess. The LArTPC technology employed by MicroBooNE provides the capability to image neutrino interactions with mm-scale precision. Computer vision and other machine learning techniques are promising tools for image processing that could boost efficiencies for selecting $\nu_{e}$-CC and other rare signals, reduce cosmic and beam-induced backgrounds, and improve the reconstruction of neutrino energies. The MicroBooNE experiment has been at the forefront of developing and testing such techniques for use in physics analyses. In this poster we overview deep-learning based reconstruction methods. We will showcase the use of a recurrent neural network to estimate neutrino energies and present a new reconstruction framework that uses convolutional neural networks to locate neutrino interaction vertices, tag pixels with track and shower labels, and perform particle identification on reconstructed clusters. We will present studies characterizing the performance of these new tools and demonstrate their effectiveness through their use in an inclusive $\nu_{e}$-CC event selection.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Beyond Binary: Automated PLC Memory Forensics through RGB Image Analysis and Deep Learning

The introduction of Industry 4.0 and the evolution of industrial control systems (ICS) to adopt Internet-based technologies enhanced productivity, but have inadvertently increased their vulnerability to cyber-based malicious attacks. When an ICS system is compromised, security analysts need to identify the root cause quickly to start the recovery process and develop mitigation strategies to safeguard against future instances. Memory forensics is critical in the analysis process to ascertain what occurred. To date, approaches to analyze the persistent memory in ICS devices are limited, and almost nonexistent for volatile memory. This paper proposes an automated methodology, COMA, for PLC memory dump analysis using computer vision and deep learning techniques. Specifically, COMA converts the sequences of bytes in a PLC memory dump to RGB pixels and creates a deep learning model that learns the underlying patterns and features of pre-labeled forensic artifacts in images and segments them into distinct regions. COMA then uses the trained model to automatically segment new memory images and extract forensic artifacts. We evaluate COMA on a Schneider Electric Modicon M221 PLC involving two cyber-based attack scenarios: (i) code injection and (ii) code modification. The empirical results show that COMA can successfully detect attack artifacts in memory dumps in both scenarios.

Asmar Awad, Rima↗
DOE OSTI2024

Demonstration and Automation of Reflected Target Optical Measurement for Heliostats

Accurate optical surfaces are a primary driver of concentrated solar power plant performance. Errors in pointing and tracking mirrors, the canting of individual mirror facets, and the surface slope of the mirror itself can be caused by errors during assembly, transportation, wind loading, gravity, and many other sources. The tools that exist to measure these error sources today largely rely on fringe deflectometry (SOFAST, QDec, others), or photogrammetry with targets attached to the mirror surface. Since 2022, NREL has been developing a measurement method called the Reflected Target Non-intrusive Assessment (ReTNA) system. This system differs from most established methods in that we perform deflectometry with a pattern of coded targets, identified in space with photogrammetry. Reflected target systems have several advantages over traditional fringe deflectometry systems. Firstly, they can be operated in bright or ambient lighting, a challenge for fringe systems that use a projector and screen. Reflected target systems also can use a much lighter and less expensive target than projector-based systems. Lastly, 2D slope measurement can be solved from a single image, which leads to several advantages for accommodating faster measurements and smaller sized targets. These advantages make ReTNA particularly well-suited for applications where there are space or lighting constraints, like performing heliostat quality assurance on an assembly line. It's also useful when a lightweight, flexible system is needed, like for heliostat developers to quickly measure a new heliostat design at different orientations, to observe gravitational effects on the mirror surface shape. In the last year, significant improvements were made to this tool to make it more useful for these applications. These improvements were focused around validation of the ReTNA measurement system, and automation of the setup and measurement process. First, we present an improved ReTNA layout, for use on the heliostat assembly line. Next, we detail the various changes to the ReTNA software and computer vision methods to automate data collection in this new setup, and lessons learned from this process. The goal with this new setup is to perform a full heliostat surface characterization without removing the mirror from the assembly line. Lastly, we share results from several ReTNA validation studies undertaken over the last year. These include repeated ReTNA measurement on demonstration mirror facets, comparisons with other optical measurement tools, and some studies aimed at quantifying the uncertainty of ReTNA measurement under various constraints (mirror-target spacing, camera resolution, etc.). These results are compared with 2024 HelioCon performance targets, and our planned next steps for the ReTNA measurement system are presented.

CSP↗
DOE OSTI2024

From Chaos to Clarity: Autonomous Materials Discovery for Extreme Environments

The pursuit of advanced functional materials for energy applications demands an understanding of their behavior under the most challenging conditions. Extreme environments, characterized by intense radiation, high temperatures, and corrosive chemistries, push materials to their limits, often revealing unexpected behaviors and degradation pathways. Traditional materials research approaches, relying on trial-and-error experimentation, are often slow and resource-intensive, ill-suited to the complexities of extreme environments. This talk will explore the transformative potential of autonomous materials science in revolutionizing our understanding of materials synthesis and degradation in extreme environments. By integrating advanced microscopy techniques, artificial intelligence, and robotic experimentation, we can accelerate the discovery and design of resilient materials for a sustainable future. The presentation will highlight recent breakthroughs in autonomous microscopy, computer vision, and machine learning, showcasing their ability to unravel complex material transformations at the atomic scale. The talk will also delve into the challenges and opportunities associated with deploying autonomous systems to probe extreme environments, emphasizing the importance of robust algorithms, real-time data analysis, and adaptive experimentation. Our ultimate goal is to empower scientists with unprecedented capabilities to explore, understand, and engineer materials that can withstand the harshest conditions, paving the way for innovations in energy, aerospace, and beyond.

artificial intelligence↗
DOE OSTI2026

Intern Poster Session 08/13: Autonomous Nuclear Robotics: Applications in nuclear waste inspection and hot cell experiments

The nuclear industry is experiencing renewed interest in autonomous robotics, yet most deployed systems remain teleoperated with limited autonomy. This work presents two contributions toward fully autonomous nuclear robotic systems: autonomous waste inspection at the Hanford Site and an autonomous hot cell laboratory framework. Inspections of Hanford's underground waste storage tanks are performed manually at significant cost and personnel exposure. We developed a reinforcement-learning (RL) training pipeline for a custom-built inspection arm. In parallel, we are designing an autonomous laboratory framework for post-irradiation examination in hot cells at the Specimen Preparation Laboratory (SPL) that integrates computer vision, task and motion planning, hardware execution, and operator-in-the-loop control. These systems demonstrate a path toward safer, more efficient nuclear operations by reducing human exposure while maintaining rigorous human oversight at critical decision points.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

BFSVBF (BatFinder Smart Video BioFilter) [SWR-22-87] and Multi-class BatFinder Smart Video BioFilter Keras

Bats are notoriously difficult to study, therefore, identifying specific behavioral trends and the precise environmental conditions at the time of collision requires a monitoring solution that can reliably collect relevant data. To date, thermal infrared video surveillance has been extensively applied to study bats and has proven to be a powerful yet cumbersome tool. Current analytical approaches are time consuming because data processing data has not been fully automated. In the past, steps have been taken to record avian and bat activity in conjunction with complicated image processing techniques that separate species from other moving objects within the field of view (i.e. clouds and portions of the wind turbine). Once the videos are collected, the post-processing does not allow real time monitoring and identification, leading to a delay in both studying the behavior of these species and determining the effectiveness of any impact reduction strategy being studied. Moreover, object identification capability is lacking, thus limiting the usefulness of video data. To resolve these issues, we are using open source computer vision and machine learning techniques allowing for automatic detection of objects in real-time with the ability to correlate these objects with environmental variables and recording the flight paths of each object. The code has gone through five rounds of development with images used to train the models. This advancement allows for automated real-time data collection, identification, and tracking, thereby eliminating the need for long and tedious post-analysis processing of the videos. We will discuss the two open source and publicly available machine learning models developed within this scope of this work: 1) a binary model with a 97.5% accuracy in identifying the difference between an object and an empty scene, including wind turbine and clouds; and 2) a multiple classification model with the capability of identifying the type of object detected: bats (90% accuracy), birds (83% accuracy), insects (69% accuracy) and non-biological (99% accuracy).

Yarbrough, John↗
DOE OSTI2024

Event-to-Video Conversion for Overhead Object Detection

Collecting overhead imagery using an event camera is desirable due to the energy efficiency of the image sensor compared to standard cameras. However, event cameras complicate downstream image processing, especially for complex tasks such as object detection. In this paper, we investigate the viability of event streams for overhead object detection. We demonstrate that across a number of standard modeling approaches, there is a significant gap in performance between dense event representations and corresponding RGB frames. We establish that this gap is, in part, due to a lack of overlap between the event representations and the pre-training data that the object detectors were initially trained on through a number of experiments. Then, apply an off-the-shelf event-to-video conversion tool that converts event streams into gray-scale video to close this gap. We demonstrate that this approach results in a large performance increase, outperforming even event-specific object detection techniques on our overhead target task. These results suggest that better aligning event representations with existing large pre-trained models may result in greater short-term performance gains compared to end-to-end event-specific architectural improvements.

machine learning (ML), computer vision, Neuromorph↗
DOE OSTI2024

A Method for Producing Hierarchical and Statistically Calibrated Predictions of Nuclear Material Properties from Existing Models

Computer vision-based analysis of micrographs of nuclear materials is an emerging technique for property prediction, synthetic route identification, and other material analysis tasks. These analysis tasks play a pivotal role in many material characterization applications such as signature development for treaty verification, process optimization, etc. The backbone in many of the recent computer vision-based techniques is a deep learning model, which takes a fixed-size set of pixels and provides a class prediction for that set of pixels. For example, previous work developed a deep convolutional neural network (CNN) to predict the synthetic route from a 256 px x 256 px patch taken from a larger image of uranium ore concentrates. In this work, we present several methods for first calibrating these models in a manner that they can provide accurate probabilities of their predictions’ veracity, and several methods of combining these probabilities. Overall, the combination of these two steps into a pipeline allows for full-image and even full-sample (where a sample has many images) predictions with associated confidence values. Finally, we show that one can also use the patch predictions and confidence to produce a visualization to map predicted constituents through the image. Results and examples for predicting and mapping uranium ore concentrates’ synthetic process from imagery will be presented.

artificial intelligence↗
DOE OSTI2026

PopGNN: Graph Neural Network-Based Flexible Future Population Forecasting Model

Accurate population forecasts is important to plan critical infrastructure and services, from housing and education to healthcare and transport. However, traditional population prediction studies have only employed traditional machine learning models limited to capture complex spatial interdependencies and patterns. Althogh recently computer vision-based framework was introduced with with promising accuracy, it has critical limitations for real-world planning applications: it function only at fixed spatial resolutions, restricting their use in diverse boundaries such as census tracts, neighborhoods, or administrative zones. Therefore, this study suggests a Graph Neural Network (GNN)-based population prediction framework, called PopGNN. This model recorded remarkable performance compared with state-of-the-art models and traditional baseline models in the grid and administrative boundaries. Furthermore, our framework achieved comparable predictive accuracy to a computer vision-based model in both the South Korea and Tennessee case studies. Consequently, this study is valuable in that a single model can provide accurate population forecasts that address diverse planning demands, ranging from granular grid-level estimates for precise service allocation and facility location planning to aggregate administrative-level forecasts for macro-scale regional policy and resource distribution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Measuring Cities with Software-Defined Sensors

The Chicago Array of Things (AoT) project, funded by the US National Science Foundation, created an experimental, urban-scale measurement capability to support diverse scientific studies. Initially conceived as a traditional sensor network, collaborations with many science communities guided the project to design a system that is remotely programmable to implement Artificial Intelligence (AI) within the devices-at the “edge” of the network-as a means for measuring urban factors that heretofore had only been possible with human observers, such as human behavior including social interaction. The concept of “software-defined sensors” emerged from these design discussions, opening new possibilities, such as stronger privacy protections and autonomous, adaptive measurements triggered by events or conditions. We provide examples of current and planned social and behavioral science investigations uniquely enabled by software-defined sensors as part of the SAGE project, an expanded follow-on effort that includes AoT.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Hands-On Computer Science: The Array of Things Experimental Urban Instrument

Chicago's Array of Things (AoT) project is aptly described as a technology experiment or a "smart city" prototype. The concept of such an extensible "instrument" arose within a larger translational research vision applying computer science and engineering research for the multidimensional benefit of people and communities in cities. The AoT project hypothesized that wireless intelligent sensor networks could enable both quantitative social science and urban monitoring while also stimulating youth interest in science and technology. Successful deployment of such sensor networks could provide open data from urban measurements not only in support of diverse research questions-in environmental dynamics, urban architecture, engineering, and social sciences-but also informing community groups and city planners. Further, the AoT project and its successor SAGE project are a computer science and engineering experiment, but its success is inextricably tied to community engagement and experiential education. Simply put, community acceptance is a prerequisite to installing and testing the instrument.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Two-Stage Wildlife Event Classification for Edge Deployment

Camera-based wildlife monitoring is often overwhelmed by non-target triggers and slowed by manual review or cloud-dependent inference, which can prevent timely intervention for high stakes human–wildlife conflicts. Our key contribution is a deployable, fully offline edge vision sensor that achieves near-real-time, highly accurate wildlife event classification by combining detector-based empty-image suppression with a lightweight classifier trained with a staged transfer-learning curriculum. Specifically, Stage 1 uses a pretrained You Only Look Once (YOLO)-family detector for permissive animal localization and empty-trigger suppression, and Stage 2 uses a lightweight EfficientNet-based binary classifier to confirm puma on detector crops and gate downstream actions. Our design is robust to low-quality nighttime monochrome imagery (motion blur, low contrast, illumination artifacts, and partial-body captures) and operates using commercially available components in connectivity-limited settings. In field deployments running since May 2025, end-to-end latency from camera trigger to action command is approximately 4 s. Ablation studies using a dataset of labeled wildlife images (pumas, not pumas) show that the two-stage approach substantially reduces false alarms in identifying pumas relative to a full-image classifier while maintaining high recall. On the held-out test set (N = 1434 events), the proposed two-stage cascade achieves precision 0.983, recall 0.975, F1 0.979, accuracy 0.986, and balanced accuracy 0.983, with only 8 false positives and 12 false negatives. The system can be easily adapted for other species, as demonstrated by rapid retraining of the second stage to classify ringtails. Downstream responses (e.g., notifications and optional audio/light outputs) provide flexible actuation capabilities that can be configured to support intervention.

58 GEOSCIENCES↗
DOE OSTI2022

Affordable Artificial Intelligence-Assisted Machine Supervision System for the Small and Medium-Sized Manufacturers

With the rapid concurrent advance of artificial intelligence (AI) and Internet of Things (IoT) technology, manufacturing environments are being upgraded or equipped with a smart and connected infrastructure that empowers workers and supervisors to optimize manufacturing workflow and processes for improved energy efficiency, equipment reliability, quality, safety, and productivity. This challenges capital cost and complexity for many small and medium-sized manufacturers (SMMs) who heavily rely on people to supervise manufacturing processes and facilities. This research aims to create an affordable, scalable, accessible, and portable (ASAP) solution to automate the supervision of manufacturing processes. The proposed approach seeks to reduce the cost and complexity of smart manufacturing deployment for SMMs through the deployment of consumer-grade electronics and a novel AI development methodology. The proposed system, AI-assisted Machine Supervision (AIMS), provides SMMs with two major subsystems: direct machine monitoring (DMM) and human-machine interaction monitoring (HIM). The AIMS system was evaluated and validated with a case study in 3D printing through the affordable AI accelerator solution of the vision processing unit (VPU).

3D printing↗
DOE OSTI2024

Using Fiducial Markers for Pose Estimation of an OSWEC in a Wave Tank: Preprint

In this study, we consider a novel method of sensing the motion of a wave energy converter during testing in a wave flume under the influence of incoming waves. The wave energy converter considered in our research is an oscillating surge wave energy converter, which is a hinged paddle that responds to incoming waves. Motion sensing is normally done with inertial sensors, which can hinder the motion due to suspended cables that carry power and transmit signals. Our proposed method is contactless and can be implemented economically. A camera is used to record different marker patterns affixed to the moving paddle and the motion deduced by pose estimation algorithms. Fiducial markers are commonly used for robot localization and in augmented reality. There are many types of fiducial markers, including ArUco-type markers which are accurate, fast and robust. The system consists of markers attached to the paddle element and recorded using a machine vision camera. A pose estimation algorithm is then applied to the detected markers to estimate the tilt of the paddle. In this work, we examine the challenges of image acquisition and calibration for underwater targets, compare the motion obtained by this new system with a calibrated tilt sensor and identify areas where the new system may be superior.

computer vision↗
DOE OSTI2024

A Vision–based Robust $\mathcal{H}$ ∞ Gain Scheduling Longitudinal and Lateral Following Controller for Autonomous Vehicles on Urban Curved Roads

Implementing advanced driver assistance systems (ADAS) in congested and intricate urban traffic scenarios poses significant challenges. To address the frequent stop–and–go motions exhibited by autonomous vehicles (AVs) navigating urban roads with changes in curvature, we propose a vision–based robust $\mathcal{H}$ ∞ adaptive cruise control system (ACC) for longitudinal control, plus a lane keeping assist system (LKAS) for lateral control. For the vision-based ACC, a weighted probability objective function for the vehicle following behavior is formulated. We incorporate $\mathcal{H}$ ∞ performance and gain scheduling techniques to mitigate the impact of uncertainty in visual sensor measurements. Furthermore, the optimal time headway is scheduled based on the velocity to ensure traffic flow efficiency and safety during the vehicle following process. For the LKAS, we introduce a road curvature estimation method that integrates lane and vehicle dynamics information to obtain the lateral and heading offsets. Next, the design criterion of the observer–based robust gain scheduling lateral motion controller is established by linear matrix inequality (LMI). Here, a series of experiments conducted within a camera–in–loop platform validate the proposed method.

33 ADVANCED PROPULSION SYSTEMS↗