SEARCH · Search NASA
Results for “image based modeling”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Critical Infrastructure Classification via CNN-based Modeling and Image Analysis
With recent advances in the fields of satellite imagery and machine learning we now have the ability to develop explainable deep learning models that enhance critical infrastructure analysis. Funded through Idaho National Laboratory’s (INL) Laboratory Directed Research and Development (LDRD) office we are in the process of developing a deep learning model capable of identifying critical infrastructure facilities and embedded features within those facilities. Utilizing current limit of practice techniques in the machine learning areas of explainability and transfer learning our model, once complete, will have the capacity to be used on multiple different imagery data sets and produce results that not only classify critical infrastructure facilities, but also explain why a critical infrastructure facility was classified as a certain type of facility. These advancements eliminate the ‘black box’ approach deep learning models have had in the past, where a user will have to trust the conclusion of a model without understanding what reasoning when into the model’s classification process. They also expand a model’s usefulness, traditionally a deep learning model will have to use the same data set it was originally trained on. Given the long training times of deep learning models this is impractical in a number of scenarios. By utilizing transfer learning advancements, we are eliminating the need to train our model on the same data set it is then run on to classify critical infrastructure facilities. We are also enabling the analysis and classification of data sets that are potentially too small to be divided into a training and testing data set. Once completed this model can provide a foundation to enhanced critical infrastructure analysis, dependency analysis, and potential disaster relief efforts.
Neural Network-Enhanced Reproducing Kernel Particle Method for Image-Based Multiphysics Damage Modeling of Energy Storage Materials
Energy storage materials undergo significant stresses during charge/discharge cycling, which makes understanding their reliability and durability fundamental in predicting performance and service life. Strong electrochemical-mechanical coupling and highly anisotropic material properties contribute to the formation and propagation of micro-cracking, largely along material interfaces and grain boundaries. With microstructural images supplied by the National Renewable Energy Laboratory (NREL), image-based modeling techniques are used to represent the complex material microstructures that dictate the coupled physics of these systems. Traditional electrochemical-mechanical models rely on mesh-based finite element methods, which can lead to difficulties in capturing crack propagation due to mesh dependency. Additionally, commonly used damage models, such as the continuous damage model and the cohesive zone model, often have steep tradeoffs between discontinuous field accuracy and computational expense. In this work, a neural network-enhanced reproducing kernel particle method (NN-RKPM) [1] is leveraged to accurately capture damage and crack propagation throughout the material by learning the location, orientation, and sharpness of discontinuity while allowing for a coarser nodal distribution than that necessary for capturing sharp solution transitions using traditional mesh-based methods. NN-RKPM is used to inform how crack opening and closure in turn affect the coupled chemical equations and material microstructure. Reference: [1] Baek, J., Chen, J. S., Susuki, K., "Neural Network enhanced Reproducing Kernel Particle Method for Modeling Localizations," International Journal for Numerical Methods in Engineering, Vol. 123, pp 4422-4454, https://doi.org/10.1002/nme.7040, 2022.
Annotation of images based on a 3D model of objects
An annotation system for providing annotations for original images is provided. In some embodiments, the annotation system accesses an annotation associated with an object of a 3D model. The annotation system also accesses and displays an original image. The annotation system renders a model image of the 3D model based on the view from which the original image was collected. When the model image contains the annotated object, the annotation system provides an indication that an annotation is associated with the object of the original image. The annotation system can provide indications of the annotation for other original images that include the annotated object irrespective of the view from which the other original images are collected.
Annotation of images based on a 3D model of objects
An annotation system for providing annotations for original images is provided. In some embodiments, the annotation system accesses an annotation associated with an object of a 3D model. The annotation system also accesses and displays an original image. The annotation system renders a model image of the 3D model based on the view from which the original image was collected. When the model image contains the annotated object, the annotation system provides an indication that an annotation is associated with the object of the original image. The annotation system can provide indications of the annotation for other original images that include the annotated object irrespective of the view from which the other original images are collected.
Improved Acquisition and Reconstruction for Wavelength-Resolved Neutron Tomography
Wavelength-resolved neutron tomography (WRNT) is an emerging technique for characterizing samples relevant to the materials sciences in 3D. WRNT studies can be carried out at beam lines in spallation neutron or reactor-based user facilities. Because of the limited availability of experimental time, potential imperfections in the neutron source, or constraints placed on the acquisition time by the type of sample, the data can be extremely noisy resulting in tomographic reconstructions with significant artifacts when standard reconstruction algorithms are used. Furthermore, making a full tomographic measurement even with a low signal-to-noise ratio can take several days, resulting in a long wait time before the user can receive feedback from the experiment when traditional acquisition protocols are used. In this paper, we propose an interlaced scanning technique and combine it with a model-based image reconstruction algorithm to produce high-quality WRNT reconstructions concurrent with the measurements being made. The interlaced scan is designed to acquire data so that successive measurements are more diverse in contrast to typical sequential scanning protocols. The model-based reconstruction algorithm combines a data-fidelity term with a regularization term to formulate the wavelength-resolved reconstruction as minimizing a high-dimensional cost-function. Using an experimental dataset of a magnetite sample acquired over a span of about two days, we demonstrate that our technique can produce high-quality reconstructions even during the experiment compared to traditional acquisition and reconstruction techniques. In summary, the combination of the proposed acquisition strategy with an advanced reconstruction algorithm provides a novel guideline for designing WRNT systems at user facilities.
Role of Imaging Specrometer Data for Model-based Cross-calibration of Imaging Sensors
Site characterization benefits from imaging spectrometry to determine spectral bi-directional reflectance of a well-understood surface. Cross calibration approaches, uncertainties, role of imaging spectrometry, model-based site characterization, and application to product validation.
Model-based VQ for image data archival, retrieval and distribution
An ideal image compression technique for image data archival, retrieval and distribution would be one with the asymmetrical computational requirements of Vector Quantization (VQ), but without the complications arising from VQ codebooks. Codebook generation and maintenance are stumbling blocks which have limited the use of VQ as a practical image compression algorithm. Model-based VQ (MVQ), a variant of VQ described here, has the computational properties of VQ but does not require explicit codebooks. The codebooks are internally generated using mean removed error and Human Visual System (HVS) models. The error model assumed is the Laplacian distribution with mean, lambda-computed from a sample of the input image. A Laplacian distribution with mean, lambda, is generated with uniform random number generator. These random numbers are grouped into vectors. These vectors are further conditioned to make them perceptually meaningful by filtering the DCT coefficients from each vector. The DCT coefficients are filtered by multiplying by a weight matrix that is found to be optimal for human perception. The inverse DCT is performed to produce the conditioned vectors for the codebook. The only image dependent parameter used in the generation of codebook is the mean, lambda, that is included in the coded file to repeat the codebook generation process for decoding.
Exploring Capability of Multimodal Foundation Model for Image-based Fault Detection of Photovoltaic Modules
Multimodal Foundation Model (MFM), like ChatGPT and Gemini, have emerged as powerful tools for their exceptional natural language processing capabilities and their emerging potential in image analysis. This paper investigates the application of MFMs for photovoltaic (PV) fault detection through image analysis, focusing on ChatGPT 4.0 and Gemini 1.5 Pro. Three types of PV images and the corresponding common PV faults are detected: bird droppings using visible images, cell cracks via electroluminescence (EL) images, and hotspots using infrared (IR) images. Among the two models, Gemini 1.5 Pro demonstrated superior performance, achieving near-perfect results with an average F1 score of 0.97, consistently outperforming ChatGPT 4.0 in accuracy and reliability. Unlike traditional machine learning (ML) models, MFMs can operate in a zero shot manner that does not require additional training by the user, and the input images are not limited by size, angle, scope, or PV technology. The strong adaptability and user-friendliness make MFM a promising tool for analyzing PV images and advancing health monitoring for PV modules.
High fidelity ultrasound imaging of concrete structures
As plans are made to extend licenses for the aging fleet of commercial nuclear power plants for periods of sixty years and beyond, research into the long-term integrity of their concrete structures has increased. Ultrasound tomography is a useful tool for nondestructive evaluation of these concrete structures. Typically, pulse-echo measurements are made over a large surface and are processed using a reconstruction algorithm, producing a 3D image of the structure that highlights embedded defects. These measurements are processed using a delay-and-sum algorithm such as the synthetic aperture focusing technique (SAFT). Oak Ridge National Laboratory is developing novel ultrasound model-based image reconstruction (U-MBIR) algorithms to improve the imaging capability of pulse-echo ultrasound array imagers. U-MBIR is an inversion technique that reconstructs the sample under test from the ultrasound measurements by formulating and solving a mathematical optimization problem using two sets of terms: one set ensures that the reconstruction matches measured data based on a model for the physics of beam propagation with the noise in the detector, and another set ensures that the reconstruction has certain properties based on a model for the object being scanned. This paper compares three techniques: SAFT, frequency-banded SAFT (FB-SAFT), and U-MBIR. The U-MBIR method produces higher quality images of the underlying concrete structures compared to the images produced using SAFT and FB-SAFT from the same set of measurements. This paper also illustrates the detection of various defects with higher confidence due to the lower noise and artifact-free U-MBIR images.
An efficient system for reliably transmitting image and video data over low bit rate noisy channels
This research project is intended to develop an efficient system for reliably transmitting image and video data over low bit rate noisy channels. The basic ideas behind the proposed approach are the following: employ statistical-based image modeling to facilitate pre- and post-processing and error detection, use spare redundancy that the source compression did not remove to add robustness, and implement coded modulation to improve bandwidth efficiency and noise rejection. Over the last six months, progress has been made on various aspects of the project. Through our studies of the integrated system, a list-based iterative Trellis decoder has been developed. The decoder accepts feedback from a post-processor which can detect channel errors in the reconstructed image. The error detection is based on the Huber Markov random field image model for the compressed image. The compression scheme used here is that of JPEG (Joint Photographic Experts Group). Experiments were performed and the results are quite encouraging. The principal ideas here are extendable to other compression techniques. In addition, research was also performed on unequal error protection channel coding, subband vector quantization as a means of source coding, and post processing for reducing coding artifacts. Our studies on unequal error protection (UEP) coding for image transmission focused on examining the properties of the UEP capabilities of convolutional codes. The investigation of subband vector quantization employed a wavelet transform with special emphasis on exploiting interband redundancy. The outcome of this investigation included the development of three algorithms for subband vector quantization. The reduction of transform coding artifacts was studied with the aid of a non-Gaussian Markov random field model. This results in improved image decompression. These studies are summarized and the technical papers included in the appendices.
A fundamental model and efficient inference for SAR ocean imagery
Employing a synthetic aperture radar (SAR) imaging model based on fundamental models of nonlinear hydrodynamics, electromagnetic scattering from a two-scale surface, and SAR imaging of a time-variant scene, the optimal (minimum mean-square error) estimates of the parameters of a sinusoidal, long gravity wave, and the short gravity wave ensemble are found in an efficient recursive form and their performance evaluated, generally by numerical simulation, in a one-dimensional stationary version. An application is made to Seasat-SAR complex imagery.
Using porous random fields to predict the elastic modulus of unoxidized and oxidized superfine graphite
Nuclear graphite is a candidate material for Generation IV nuclear power plants. Porous materials such as graphite can contain complex networks of pores that influence the material's mechanical and irradiation response. A methodology known as the random finite element method (RFEM) was adapted to create synthetic microstructures and predict the influence of porosity on the elastic properties of graphite during oxidation. RFEM combines random field theory and the finite element method in a Monte Carlo framework to estimate the mechanical response of a given grade of graphite. In this research, the random fields were verified through experimental characterization to predict the elastic response of three nuclear graphite grades, ETU-10, IG-110, and 2114. Finite element models (FEM) were generated using segmentations of x-ray computed tomography (XCT) data known as image-based models (IBMs) to validate and compare with the RFEM results and better understand the effects of uniform oxidation in these graphite grades. The RFEM predictions appear to correlate well with the experimental values of the measured Young’s modulus of the three graphite grades and display the same trends as IBMs.
Hydrogen bonded trimesic acid networks on Cu(111) reveal how basic chemical properties are imprinted in HR-AFM images
High resolution non-contact atomic force microscopy measurements characterize assemblies of trimesic acid molecules on Cu(111) and the link group interactions, providing the first fingerprints utilizing CO-based probes for this widely studied paradigm for hydrogen bond driven molecular self assembly. The enhanced submolecular resolution offered by this technique uniquely reveals key aspects of the competing interactions. Accurate comparison between full-density-based modeled images and experiment allows to identify key structural elements in the assembly in terms of the electron-withdrawing character of the carboxylic groups, interactions of those groups with Cu atoms in the surface, and the valence electron density in the intermolecular region of the hydrogen bonds. This study of trimesic acid assemblies on Cu(111) combining high resolution atomic force microscopy measurements with theory and simulation forges clear connections between fundamental chemical properties of molecules and key features imprinted in force images with submolecular resolution.
A machine learning decision criterion for reducing scan time for hyperspectral neutron computed tomography systems
We present the first machine learning-based autonomous hyperspectral neutron computed tomography experiment performed at the Spallation Neutron Source. Hyperspectral neutron computed tomography allows the characterization of samples by enabling the reconstruction of crystallographic information and elemental/isotopic composition of objects relevant to materials science. High quality reconstructions using traditional algorithms such as the filtered back projection require a high signal-to-noise ratio across a wide wavelength range combined with a large number of projections. This results in scan times of several days to acquire hundreds of hyperspectral projections, during which end users have minimal feedback. To address these challenges, a golden ratio scanning protocol combined with model-based image reconstruction algorithms have been proposed. This novel approach enables high quality real-time reconstructions from streaming experimental data, thus providing feedback to users, while requiring fewer yet a fixed number of projections compared to the filtered back projection method. In this paper, we propose a novel machine learning criterion that can terminate a streaming neutron tomography scan once sufficient information is obtained based on the current set of measurements. Our decision criterion uses a quality score which combines a reference-free image quality metric computed using a pre-trained deep neural network with a metric that measures differences between consecutive reconstructions. The results show that our method can reduce the measurement time by approximately a factor of five compared to a baseline method based on filtered back projection for the samples we studied while automatically terminating the scans.
Model-based reconstruction for enhanced x-ray CT of dense tri-structural isotropic particles
Tri-structural isotropic (TRISO) fuel particles are a key component of next generation nuclear fuels. Using x-ray computed tomography (CT) to characterize TRISO particles is challenging because of the strong attenuation of the x-ray beam by the uranium core, leading to severe photon starvation in a substantial fraction of the measurements. Furthermore, the overall acquisition time for a high-resolution CT scan can be very long when using conventional laboratory-based x-ray systems and reconstruction algorithms. Specifically, when analytic methods such as the Feldkamp–Davis–Kress (FDK) algorithm are used for reconstruction, it results in severe streak artifacts and noise in the corresponding 3D volume, which makes subsequent analysis of the particles challenging. In this paper, we develop and apply model-based image reconstruction (MBIR) algorithms to improve the quality of CT reconstructions for TRISO particles to facilitate better characterization. We demonstrate that the proposed MBIR algorithms can significantly suppress artifacts with minimal pre-processing compared to conventional approaches. We also demonstrate that the proposed MBIR approach can obtain high-quality reconstruction compared to the FDK approach even when using a fraction of the typically acquired measurements, thereby enabling dramatically faster measurement times for TRISO particles.
Location generalizability of image-based air quality models
This paper is to be submitted at the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Computer Vision for Earth Observation workshop. The full paper abstract is below: The ability to rapidly quantify atmospheric pollutants is important both for global emissions monitoring and for mitigating the adverse effects that follow a hazardous chemical release. In the aftermath of a chemical release, imagery is often the only available resource to assess local conditions. Recent work has demonstrated initial success in predicting particulate matter pollution from imagery; however, these results are tied to a specific site and do not generalize to new geographic locations. In this work, we seek to understand how easily deep learning models generalize to new locations in the context of image-based air quality assessments, targeting two distinct tasks: (1) broad measures of particulate matter pollution, and (2) the mass of a given chemical released in hazardous plumes. For the latter, we focus on sulfur dioxide, a toxic aerosol and a major component of particulate matter pollution caused by industrial fossil fuel consumption. To develop a model that operates in the widest possible range of environments, we test different training strategies, including the use of new geolocation foundation models. The best performing models achieve >80% accuracy when evaluating unseen imagery at previously seen sites, but we find significant drops in performance when evaluating imagery from unseen sites, at best 65%. Additionally, we present the public release of the National Parks Air Quality Index Dataset, a new medium-sized dataset that pairs imagery with sensor-based air quality measurements at 15 different national parks.
Computer Model Calibration Based on Image Warping Metrics: An Application for Sea Ice Deformation
Arctic sea ice plays an important role in the global climate. Sea ice models governed by physical equations have been used to simulate the state of the ice including characteristics such as ice thickness, concentration, and motion. More recent models also attempt to capture features such as fractures or leads in the ice. These simulated features can be partially misaligned or misshapen when compared to observational data, whether due to numerical approximation or incomplete physics. In order to make realistic forecasts and improve understanding of the underlying processes, it is necessary to calibrate the numerical model to field data. Traditional calibration methods based on generalized least-square metrics are flawed for linear features such as sea ice cracks. We develop a statistical emulation and calibration framework that accounts for feature misalignment and misshapenness, which involves optimally aligning model output with observed features using cutting-edge image registration techniques. Here, this work can also have application to other physical models which produce coherent structures.