Search NASA⌕ Search

SEARCH · Search NASA

Results for “Autonomous Experimentation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Towards developing multiscale-multiphysics models and their surrogates for digital twins of metal additive manufacturing

Artificial intelligence (AI) embedded within digital models of manufacturing processes can be used to improve process productivity and product quality significantly. The application of such advanced capabilities particularly to highly digitalized processes such as metal additive manufacturing (AM) is likely to make those processes commercially more attractive. AI capabilities will reside within Digital Twins (DTs) which are living virtual replicas of the physical processes. DTs will be empowered to operate autonomously in a diagnostic control capacity to supervise processes and can be interrogated by the practitioner to inform the optimal processing route for any given product. The utility of the information gained from the DTs would depend on the quality of the digital models and, more importantly, their faster-solving surrogates which dwell within DTs for consultation during rapid decision-making. In this article, we point out the exceptional value of DTs in AM and focus on the need to create high-fidelity multiscale-multiphysics models for AM processes to feed the AI capabilities. We identify technical hurdles for their development, including those arising from the multiscale and multiphysics characteristics of the models, the difficulties in linking models of the subprocesses across scales and physics, and the scarcity of experimental data. We discuss the need for creating surrogate models using machine learning approaches for real-time problem-solving. We further identify non-technical barriers, such as the need for standardization and difficulties in collaborating across different types of institutions. We offer potential solutions for all these challenges, after reflecting on and researching discussions held at an international symposium on the subject in 2019. Here, we argue that a collaborative approach can not only help accelerate their development compared with disparate efforts, but also enhance the quality of the models by allowing modular development and linkages that account for interactions between the various sub-processes in AM. A high-level roadmap is suggested for starting such a collaboration.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Toward machine-learning-assisted PW-class high-repetition-rate experiments with solid targets

We present progress in utilizing a machine learning (ML) assisted optimization framework to study the trends in a parameter space defined by spectrally shaped, high-intensity, petawatt-class (8 J, 45 fs) laser pulses interacting with solid targets and give the first simulation-based overview of predicted trends. A neural network (NN) incorporating uncertainty quantification is trained to predict the number of hot electrons generated by the laser–target interaction as a function of pulse shaping parameters. The predictions of this NN serve as the basis function for a Bayesian optimization framework to navigate this space. For post-experimental evaluation, we compare two separate neural network (NN) models. One is based solely on data from experiments, and the other is trained only on ensemble particle-in-cell simulations. Reviewing the predicted and observed trends across the experiment-capable laser parameter search space, we find that both ML models predict a maximal increase in hot electron generation at a level of approximately 12%–18%; however, no statistically significant enhancement was observed in experiments. On direct comparison of the NN models, the average discrepancy is 8.5%, with a maximum of 30%. Since shot-to-shot fluctuations in experiments affect the observations, we evaluate the behavior of our optimization framework by performing virtual experiments that vary the number of repeated observations and the noise levels. Here, we discuss the implications of such a framework for future autonomous exploration platforms in high-repetition-rate experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Probe microscopy is all you need *

We pose that microscopy offers an ideal real-world experimental environment for the development and deployment of active Bayesian and reinforcement learning methods. Indeed, the tremendous progress achieved by machine learning (ML) and artificial intelligence over the last decade has been largely achieved via the utilization of static data sets, from the paradigmatic MNIST to the bespoke corpora of text and image data used to train large models such as GPT3, DALL·E and others. However, it is now recognized that continuous, minute improvements to state-of-the-art do not necessarily translate to advances in real-world applications. We argue that a promising pathway for the development of ML methods is via the route of domain-specific deployable algorithms in areas such as electron and scanning probe microscopy and chemical imaging. This will benefit both fundamental physical studies and serve as a test bed for more complex autonomous systems such as robotics and manufacturing. Favorable environment characteristics of scanning and electron microscopy include low risk, extensive availability of domain-specific priors and rewards, relatively small effects of exogenous variables, and often the presence of both upstream first principles as well as downstream learnable physical models for both statics and dynamics. Recent developments in programmable interfaces, edge computing, and access to application programming interfaces (APIs) facilitating microscope control, all render the deployment of ML codes on operational microscopes straightforward. We discuss these considerations and hope that these arguments will lead to create novel set of development targets for the ML community by accelerating both real world ML applications and scientific progress.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Invariant discovery of features across multiple length scales: Applications in microscopy and autonomous materials characterization

Physical imaging is a foundational characterization method in areas from condensed matter physics and chemistry to astronomy and spans length scales from atomic to universe. Images encapsulate crucial data regarding atomic bonding, materials microstructures, and dynamic phenomena such as microstructural evolution and turbulence, among other phenomena. The challenge lies in effectively extracting and interpreting this information. Variational Autoencoders (VAEs) have emerged as powerful tools for identifying the underlying factors of variation in image data, providing a systematic approach to distilling meaningful patterns from complex data sets. However, a significant hurdle in their application is the definition and selection of appropriate descriptors reflecting local structures. Here, we introduce the scale-invariant VAE approach (SI-VAE) based on the progressive training of the VAE with the descriptors sampled at different length scales. The SI-VAE allows the discovery of the length scale-dependent factors of variation in the system. Here, we illustrate this approach using the ferroelectric domain images and generalize it to the movies of the electron-beam induced phenomena in graphene and topography evolution across combinatorial libraries. This approach can further be used to initialize the decision making in automated experiments including structure–property discovery and can be applied across a broad range of imaging methods. This approach is universal and can be applied to any spatially resolved data including both experimental imaging studies and simulations, and can be particularly useful for exploration of phenomena such as turbulence and scale-invariant transformation fronts.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Real-time Simulation Framework for Hardware-in-the-Loop Testing of Multi-port Autonomous Reconfigurable Solar Power Plant (MARS)

Multiport autonomous reconfigurable solar power plant (MARS) has been proposed for integrated development of photovoltaic (PV) and energy storage system (ESS) that can connect to high-voltage direct current (HVdc) and alternating current (ac) transmission grid. To de-risk the development of this complex integrated system that consists of hundreds to thousands of power electronics modules, a controller hardware-in-the-loop (cHIL) test setup will be extremely beneficial. The cHIL testing can be used for evaluation of modules as well as the hierarchical control system in MARS. With the unique configuration of power electronics modules in MARS, it becomes necessary to develop custom-designed real-time simulation models in the cHIL setup in absence of off-the-shelf models. In this paper, high-fidelity dynamic model of MARS, control algorithms at the lower level, and required communication algorithms are developed and optimized for real-time performance in the cHIL setup. Real-time experimental results from the cHIL are provided.

Dong, Zerui↗
DOE OSTI2025

Active Learning Surrogates for Integrating Electron Microscopy and Computational Insights from Simulations in Autonomous Experiments

Artificial Intelligence (AI) combined with simulations and experiments has great potential to accelerate scientific discovery across technology and pharmaceuticals. However, the gap between simulations and experiments is challenging due to disparities in time and scale, making it difficult to estimate properties like energy and electronic states from experiments, and to provide feedback based on theoretical insights.Our research addresses the challenge by developing unique deep kernel based surrogate models that learns from microscopic images, mapping structural features to energy differences from defect formation. We start with full-training using simulated images to determine optimal settings, establishing a baseline for active learning. Using these settings from the baseline, active learning is trained, and predicts structures along simulation trajectories based on uncertainty and energetic stability, thus reducing data requirements, simulation time and computational costs. The results demonstrate that the model achieves a low average error margin of approximately 0.03 meV, indicating good performance. To enhance feature extraction and reconstruction capabilities, we developed an autoencoder-decoder as additional surrogate to create latent space to capture essential features, enabling precise comparisons between simulations and experiments. The results from this model achieved a reconstruction loss of around 0.2 and accurately reconstructed molecular structures.Overall, this work advances the steering of experiments through computational simulations by employing a surrogate models that actively predicts the trajectories of structural evolution, achieving time-to-solution comparable to experimental measurements.

Saranathan, Gayathri [Hewlett-Packard]↗
DOE OSTI2022

Inverter Reliability Estimation for Advanced Inverter Functionality

In the near future, grid operators are expected to regularly use advanced distributed energy resource (DER) functions, defined in IEEE 1547-2018, to perform a range of grid-support operations. Many of these functions adjust the active and reactive power of the device through commanded or autonomous modes, which will produce new stresses on the grid-interfacing power electronics components, such as DC/AC inverters. In previous work, multiple DER devices were instrumented to evaluate additional component stress under multiple reactive power setpoints. We utilize quasi-static time-series simulations to determine voltage-reactive power mode (volt-var) mission profile of inverters in an active power system. Mission profiles and loss estimates are then combined to estimate the reduction of the useful life of inverters from different reactive power profiles. It was found that the average lifetime reduction was approximately 0.15% for an inverter between standard unity power factor operation and the IEEE 1547 default volt-var curve based on thermal damage due to switching in the power transistors. For an inverter with an expected 20-year lifetime, the 1547 volt-var curve would reduce the expected life of the device by 12 days. This framework for determining an inverter's useful life from experimental and modeling data can be applied to any failure mechanism and advanced inverter operation.

component degradation↗
DOE OSTI2023

Modular HF Isolated MV String Inverters Enable a New Paradigm for Large PV Farms

The “Modular HF Isolated MV String Inverters Enable a New Paradigm for Large PV Farms” project focuses on exploring alternative power converter and system-level plant configurations to achieve the lowest cost and highest energy output for a given solar plus storage (e.g., PV + battery) plant, including the use of medium voltage (MV) collection while taking into account detailed models of all elements. To realize this objective, four approaches were utilized (i) employ a novel Medium Voltage String Inverter (MVSI) topology (soft switching solid state transformer – S4T) to convert 1000 Vdc to 4.16 kVac; (ii) plant collection using standard, low-cost overhead MV distribution network; (iii) enable energy storage integration without additional converter cost to achieve dispatchability of the PV resource; and (iv) provide advanced functionality (autonomous operation, track ISO signals for dynamic balancing and ancillary services, and PV farm operation as a virtual grid resource). Subsequently and in alignment with the previously mentioned approaches, the project was structured in five efforts (i) S4T MVSI simulation and design; (ii) system analysis and storage optimization; (iii) financial analysis; (iv) power converter prototype build and test; and (v) regulatory and commercial impact study. The outcomes provided by each effort can be summarized as follows (i) Project explored the use of MV AC distribution architecture for hybrid PV+storage utility-scale PV farms; (ii) Detailed loss and LCOE analysis for AC and DC side BESS architecture, including multiple converter topologies, as well as for proposed MVSI/MDCT systems; (iii) MVSI was built and holds promise but needs lower-cost high-voltage Si-C devices, which does not seem possible in the near term; (iv) MDCT provides a simpler modular building block – validated through HIL and farm level modeling, simulation and experimental validation; (v) 300 kVA MDCT prototype built and tested, technology is being commercialized; and (vi) Regulatory model of utility building PV plants, where PV panels are treated as DC generation (IPP), seems viable and can allow improved grid integration.

14 SOLAR ENERGY↗
DOE OSTI2025

AI-Optimized Polarization at Jefferson Lab

The AI-Optimized Polarization project seeks to develop experimental control applications for polarized targets and beams at Jefferson Lab using AI/ML. This paper will focus on two ongoing efforts involving a cryogenic polarized target and a linearly-polarized photon beam. Firstly, cryogenic targets, such as those used in Halls B and C (and approved for Hall D), are complex systems that are sensitive to a number of factors, including the temperature, beam currents, and the microwave and NMR apparatus. Secondly, the Hall D photon beam polarization depends on the optimal orientation of a diamond radiator, which produces coherent bremsstrahlung radiation from the electron beam incident upon it. Manual operation of both systems is tedious and error prone; implementing well-designed, interpretable control systems that incorporate AI is expected to lead to improved real-time polarization. AI optimization of nuclear physics experiments will lead, not just to cost-savings, but also to more efficient and higher-quality data, and this project will help to lay the foundation for future autonomous experiments.

Moran, Patrick [College of William and Mary, Willi↗
DOE OSTI2023

Downhole Sensing and Event-Driven Sensor Fusion for Depth-of-Cut Based Autonomous Fault Response and Drilling Optimization

Achieving robust and efficient drilling is a critical part of reducing the cost of geothermal energy exploration and extraction. Drilling performance is often evaluated using one or more of three key metrics: depth of cut (DOC), rate of penetration (ROP), and mechanical specific energy (MSE). All three of these quantities are related to each other. DOC refers to the depth a bit penetrates into rock during drilling. This is an important quantity for estimating bit behavior. ROP is the simply the DOC multiplied by the rotational rate, and represents how quickly the drill bit is advancing through the ground. ROP is often the parameter used for drilling control and optimization. Finally, MSE provides insight into drilling efficiency and rock type. MSE calculations rely on ROP, drilling force, and drilling torque. Surface-based sensors at the top of the drill are often used to measure all these quantities. However, top-hole measurements can deviate substantially from the behavior at the bit due to lag, vibrations, and friction. Therefore, relying only on top-hole information can lead to suboptimal drilling control. In this work, we describe recent progress towards estimating ROP, DOC, and MSE using down-hole sensing. We assume down-hole measurements of torque, weight-on-bit (WOB). Our hypothesis is that these measurements can provide more rapid and accurate measures of drilling performance. We show how a multi-layer perceptron (MLP) machine learning algorithm can provide rapid and accurate performance when evaluated on experimental data taken from Sandia’s Hard Rock Drilling Facility. In addition, we implement our algorithms on an embedded system intended to emulate a bottom-hole-assembly for sensing and estimation. Our experimental results show that DOC can be estimated accurately and in real-time. These estimates when combined with measurements for rotary speed, torque, and force can provide improved estimates for ROP and MSE. These results have the potential to enable better drilling assessment, improved control, and extended component lifetimes.

15 GEOTHERMAL ENERGY↗
DOE OSTI2025

Leveraging unlabeled SEM datasets with self-supervised learning for enhanced particle segmentation

Scanning Electron Microscopes (SEMs) are widely used in experimental science laboratories, often requiring cumbersome and repetitive user analysis. Automating SEM image analysis processes is highly desirable to address this challenge. In particle sample analysis, Machine Learning (ML) has emerged as the most effective approach for particle segmentation. However, the time-intensive process of manually annotating thousands of SEM images limits the applicability of supervised learning approaches. Self-Supervised Learning (SSL) offers a promising alternative by enabling knowledge extraction from raw, unlabeled data. This study presents a framework for evaluating SSL techniques in SEM image analysis, focusing on novel methods leveraging the ConvNeXtV2 architecture for particle detection. A dataset comprising 25,000 SEM images is curated to benchmark these proposed SSL methods. The results demonstrate that ConvNeXtV2 models, with varying parameter counts, consistently outperform other techniques in particle detection across different length scales, achieving up to a 34% reduction in relative error compared to established SSL methods. Furthermore, an ablation study explores the relationship between dataset size and SSL performance, providing actionable insights for practitioners regarding model selection and resource efficiency. This research advances the integration of SSL into autonomous analysis pipelines and supports its application in accelerating materials science discovery.

Rettenberger, Luca↗
DOE OSTI2022

Multi-Physics Investigation of a Natural Circulation Molten Salt Micro-Reactor that Utilizes an Experimental In-Pile Device to Improve Core Physics and System Thermal-Hydraulic Performance.

INL employee PhD Dissertation - The Molten Salt Reactor (MSR) concept is a rapidly evolving Generation IV design that has recently attracted favorable attention due to the potential for reducing waste generation, realizing passive safety features, and seizing on the opportunity for cost effective economics. This thesis investigates the performance benefit of a new device invented by the doctoral candidate. The device is referred to as a Wrapped Helix around an Inclined Plane or WHIP. The WHIP is protected under a provisional patent filed with the USA Patent Office on 28 September 2021 under application number 63/261,776, BEA docket number BA-1254. The WHIP device can be located in-core or near-core to promoted enhanced thermal-hydraulics and neutronics performance. While the WHIP can be employed in a variety of solid or liquid fuel designs, this thesis investigates the device’s benefit in the application of a natural-circulation, micro-molten salt nuclear battery concept (MsNB). This thesis will specifically investigate the temperature coefficient of the MsNB fuel (FLiNaK) using novel temperature sensitivity techniques unique to Serpent particle transport code, evaluate the WHIP’s thermal-hydraulic and neutronic performance effects using both established (STAR-CCM+) and novel (Python code developed to estimate the circulation effective delayed neutron fraction, ßef f ) analytical and numerical methods, evaluate the neutron noise behavior of the MsNB and how the WHIP may alter the character of the MsNB’s transfer function, and how the WHIP affects the autonomous, load-following performance under transient power conditions using Python code developed by the candidate. The sum body of this work, in part, has been published in three journal articles as the timing of the provisional patent process has allowed. Results show that creative utilization of WHIP engineering design and function reduces reactor system volume, fuel loading, control/stability in the buoyant regime

buoyant flow↗
DOE OSTI2026

Leveraging large language models to address data scarcity in machine learning for graphene synthesis

Machine learning in experimental materials science faces significant challenges due to the scarcity of data, which are costly and time-consuming to generate, particularly when relying on in-house experiments. Literature data mining offers a potential solution but introduces issues like mixed data quality, inconsistent formats, and non-uniform reporting of synthesis parameters, resulting in partially missing and heterogeneous features across the dataset. Here, we propose data imputation and feature engineering methods that employ pre-trained large language models (LLMs) to enhance machine learning performance on scarce, heterogeneous datasets, demonstrated on graphene CVD synthesis data and the ML-HydPARK hydrogen storage dataset. GPT models perform data imputation via tailored prompting and semantic normalization of inconsistently reported features through embeddings, for example, to harmonize the complex nomenclature of CVD substrates. Beyond yielding more diverse and richer feature representations than traditional methods such as K-nearest neighbors (KNN) and Multivariate Imputation by Chained Equations (MICE), LLM-based data imputation is evaluated against dataset characteristics and prompting strategies. We vary the level of autonomy granted to the LLM, from generic prompting that leverages pre-trained knowledge for autonomous data generation to data-informed prompting that constrains outputs using target-specific information, and demonstrate which level of autonomy yields superior imputation performance across datasets and feature types. The proposed data engineering methods markedly improve downstream performance; for example, in graphene layer number classification using a support vector machine (SVM), binary accuracy increases from 39% to 65% and ternary accuracy from 52% to 72%. Fine-tuning experiments on both datasets show that combining our proposed LLM-based data imputation and feature encoding methods with numerical machine learning predictors outperforms standalone fine-tuned LLM predictors in data-scarce settings. The proposed strategies emphasize data enhancement techniques rather than refining learning architectures or regularizing loss functions, offering a broadly applicable framework for improving machine learning performance on scarce, inhomogeneous datasets.

Chemical vapor deposition↗
DOE OSTI2025

A New Vehicle-to-Vehicle Communication System: Visual-Enhanced Cooperative Traffic Operations

The advent of Connected and Autonomous Vehicles (CAVs) has highlighted the necessity for robust communication systems between vehicles and their environment. This study introduces a novel vehicle-to-vehicle (V2V) communication system, termed the Visual-Enhanced Cooperative Traffic Operations (VECTOR) system. The VECTOR system addresses the need for robust communication by converting dynamic data (including velocity and yaw angle data) into binary code, which is displayed on an LED panel mounted on the top of the vehicle. Following vehicles detect this panel and decode the information using a camera, implementing a visual-based communication method. VECTOR system employs a comprehensive five-module process. Initially, polynomial fitting techniques are applied to velocity data over fixed time intervals using third-degree polynomials, with validation via R² and MSE metrics. The second module converts velocity and yaw angle data into binary form, thereby enhancing detection and processing efficiency. The third module focuses on improving detection stability across various environmental conditions to enhance traffic safety. The fourth module decodes the binary data back into trajectory information, ensuring the fidelity of velocity and yaw angles. The final module integrates eco-control through the VECTOR system, employing advanced control algorithms to minimize energy consumption in CAVs. Experimental evaluations conducted using a modified CAV test platform based on the Lincoln MKZ demonstrate the feasibility and efficiency of the VECTOR system, achieving a 75% R-squared accuracy rate in replicating original velocity data. This methodology not only highlights potential applications but also underscores significant implications for advancing CAV technology.

Ma, Ke↗
DOE OSTI2021

Transforming Energy through Computational Science: Computing for Clean Energy

This fact sheet discusses the opportunity space for NREL's computational science capabilities to address national clean energy objectives. Achieving a carbon-free power sector by 2035 as a step towards a decarbonized U.S. energy economy in 2050 will require major advances in power generation, autonomous energy systems, transportation, and buildings/communities. Development of integrated modeling approaches for complex energy systems will be essential for deployment. Success requires developments in optimization and control theory, complemented by machine learning (ML) and artificial intelligence (AI), all of which in turn need targeted investments in breadth and scale of computing. This document defines an opportunity space where: embracing computing can link established research and development (R&D) to a decarbonization agenda; pursuing emerging approaches can accelerate the pace of technology advancement across the portfolio; and leading by example could reduce the carbon footprint of computing worldwide.

advanced computing↗
DOE OSTI2024

Evaluating direct vessel injection accident-event progression of AP1000 and key figures of merit to support the design and development of water-cooled small modular reactors

The passive safety systems (PSSs) within water-cooled reactors are meticulously engineered to function autonomously, requiring no external power source or manual intervention. They depend exclusively on inherent natural forces and the fundamental principles of reactor physics, such as gravity, natural convection, and phase changes, to manage, alleviate, and avert the release of radioactive materials into the environment during accident scenarios like a loss-of-coolant accident (LOCA). PSSs are already integrated into such operating commercial reactors as the Advanced Pressurized Reactor-1000 MWe (AP1000) and the Water-Water Energetic Reactor-1200 MWe (WWER-1200) are adopted in most of the upcoming small modular reactor (SMR) designs. Examples of water-cooled SMR PSSs are the passive emergency core-cooling system (ECCS), passive containment cooling system (PCCS), and passive decay-heat removal system, the designs of which vary based on reactor system-design requirements. However, understanding the accident-event progression and phases of a LOCA is pivotal for adopting a specific PSS for a new SMR design. This study covers the accident-event progression for direct vessel injection (DVI) small-break loss-of-coolant accident (SB-LOCA), associated physics phenomena, knowledge gaps, and important figures of merit (FOMs) that may need to be evaluated and assessed to validate thermal-hydraulics models with an available experimental dataset to support new SMR design and development.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

An Intelligent Garbage Sorting System Based on Edge Computing and Visual Understanding of Social Internet of Vehicles

In order to enable Social Internet of Vehicles devices to achieve the purpose of intelligent and autonomous garbage classification in a public environment, while avoiding network congestion caused by a large amount of data accessing the cloud at the same time, it is therefore considered to combine mobile edge computing with Social Internet of Vehicles to give full play to mobile edge computing features of high bandwidth and low latency. At the same time, based on cutting-edge technologies such as deep learning, knowledge graph, and 5G transmission, the paper builds an intelligent garbage sorting system based on edge computing and visual understanding of Social Internet of Vehicles. First of all, for the massive multisource heterogeneous Social Internet of Vehicles big data in the public environment, different item modal data adopts different processing methods, aiming to obtain a visual understanding model. Secondly, using the 5G network, the model is deployed on the edge device and the cloud for cloud-side collaborative management, aiming to avoid the waste of edge node resources, while ensuring the data privacy of the edge node. Finally, the Social Internet of Vehicles devices is used to make intelligent decision-making on the big data of the items. First, the items are judged as garbage, and then the category is judged, and finally the task of grabbing and sorting is realized. The experimental results show that the system proposed in this paper can efficiently process the big data of Social Internet of Vehicles and make valuable intelligent decisions. At the same time, it also has a certain role in promoting the promotion of Social Internet of Vehicles devices.

Shen, Xuehao↗
DOE OSTI2022

Capsule network-based semantic segmentation model for thermal anomaly identification on building envelopes

Thermography technology is widely used to inspect thermal anomalies in building façade systems. Computer vision-based techniques provide opportunities to autonomously detect such heat anomalies to significantly improve the efficiency of decision-making for building envelope retrofitting and maintenance. Here, in this work, we propose a novel Capsule Network-based deep learning model – CapsLab – that detects and identifies thermal anomalies by semantic segmentation. CapsLab is built based on our proposed prediction-tuning capsule (PT-Capsule) layer. Different from a traditional capsule layer, which consists of part-whole transformation and capsule-routing process, the proposed layer is composed of a prediction and tuning process, which helps decreasing the number of model parameters significantly. While the applicability of traditional Capsule Networks (CapsNets) has been limited to simpler tasks and smaller datasets due to their scalability issue, we can leverage the lightweight of the proposed PT-Capsule layer, and apply it to the semantic segmentation task. In this work, we also employ our previously presented performance metric, referred to as the Anomaly Identification Metric (AIM) (Kakillioglua et al. 2021), to evaluate the segmentation outputs. Traditional performance metrics do not accurately reflect the true performance of the segmentation models in thermal anomaly identification due to the high subjectivity in the annotation process and higher overlap ratio sensitivity of the standard metrics. AIM, on the other hand, is robust to these drawbacks. Experimental results show, both qualitatively and quantitatively, that our proposed segmentation method can effectively segment the thermal anomalies. Specifically, our model provides 9.38% and 13.53% improvements over the baseline model – DeepLabV3+ – based on traditional mIoU score and the AIM score, respectively, while requiring less model parameters and less computation at the same time. In addition, the scores that the AIM metric generates better align with the scores provided by building performance experts.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗