Search NASA⌕ Search

SEARCH · Search NASA

Results for “Image Transforms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Advanced Photovoltaic Module Characterization: Using Image Transformers for Current–Voltage Curve Prediction From Electroluminescence Images

Individual photovoltaic (PV) module health monitoring can be a daunting task for operation and maintenance of solar farms. Modules can be inspected through luminescence, thermal imaging, and current–voltage (I–V) curve analyzes for identification of damage and power loss. I–V curves provide easily interpretable data to determine module health as they directly provide electrical performance metrics. However, in order to obtain these curves, modules must be disconnected from the array and either removed to a solar simulator or characterized in situ with corrections for module temperature, the incident solar spectrum, and intensity. Luminescence or thermal images of a module are relatively easy to acquire in situ. Electroluminescence (EL) images highlight physical defects in the modules but do not provide easily interpretable features to correlate with electrical performance. This work presents a SWin transformer network to predict I–V curves for PV modules from their corresponding EL images. The predicted I–V curves allow the accurate prediction of the maximum power point (MPP), short-circuit current I sc , and open-circuit voltage V oc with a mean error less of than 1%. Comparing single diode model (SDM) parameters extracted from the predicted curves to those extracted from the true curves, the series resistance R s demonstrates a mean error of 5.19%, and the photocurrent I a mean error of 0.197%. The shunt resistance R sh and dark current Io parameters are predicted with larger errors because of their sensitivity to small changes in the I–V curve.

Byford, Brandon K. [New Mexico State Univ., Las Cr↗
DOE OSTI2025

Sequence length scaling in vision transformers for scientific images on frontier

Vision Transformers (ViTs) are pivotal for foundational models in scientific imagery, including Earth science applications, due to their capability to process large sequence lengths. While transformers for text have inspired scaling sequence lengths in ViTs, adapting these for ViTs introduces unique challenges. We develop distributed sequence parallelism for ViTs, enabling them to handle up to 1M tokens. Our approach, leveraging DeepSpeed-Ulysses and Long-Sequence-Segmentation with model sharding, is the first to apply sequence parallelism in ViT training, achieving a 94% batch scaling efficiency on 2,048 AMD-MI250X GPUs. Evaluating sequence parallelism in ViTs, particularly in models up to 10B parameters, highlighted substantial bottlenecks. We countered these with hybrid sequence, pipeline, and flash attention strategies, to scale beyond single GPU memory limits. Our method significantly enhances climate modeling accuracy by 20% in temperature predictions, marking the first training of a vision transformer model to convergence with a sequence length of 188K tokens, using full self-attention.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗
DOE OSTI2024

Adaptive Patching for High-resolution Image Segmentation with Transformers

Attention-based models are proliferating in the space of image analytics, including segmentation. The standard method of feeding images to transformer encoders is to divide the images into patches and then feed the patches to the model as a linear sequence of tokens. For high-resolution images, e.g. microscopic pathology images, the quadratic compute and memory cost prohibits the use of an attention-based model, if we are to use smaller patch sizes that are favorable in segmentation. The solution is to either use custom complex multi-resolution models or approximate attention schemes. We take inspiration from Adapative Mesh Refinement (AMR) methods in HPC by adaptively patching the images, as a pre-processing step, based on the image details to reduce the number of patches being fed to the model, by orders of magnitude. This method has a negligible overhead, and works seamlessly with any attention-based model, i.e. it is a pre-processing step that can be adopted by any attention-based model without friction. We demonstrate superior segmentation quality over SoTA segmentation models for realworld pathology datasets while gaining a geomean speedup of 6.9× for resolutions up to 64K2, on up to 2, 048 GPUs.

Zhang, Enzhi↗
DOE OSTI2025

Comparative Analysis of Radial and Random Microstructures of Mesophase Pitch Carbon Fibers

Carbon fibers (CF) with radial and random microstructures are produced. Here, these fibers are subjected to identical treatment before being mechanically tested and analyzed with Weibull analysis, with the results revealing a statistically significant difference in tensile strengths of 2.23 GPa for random CF and 1.69 GPa for radial CF. Raman mapping probed the crystalline structure perpendicular to the fiber axis and found a uniform structure, while wide‐angle X‐ray diffraction showed a significant difference of 7.5 Å in the crystallites’ basal lengths parallel to the fiber. Small‐angle X‐ray scattering is completed parallel to the fiber for the first time. A cross‐section Guinier plot of the 1D azimuthal integration is generated assuming symmetric scattering, and the parallel scatterers are found to have a similar length scale to the crystallite's length, validating the testing method. Finally, transmission electron microscopy is completed on the longitudinal cross‐section of each fiber. The radial carbon fiber is found to have a core–shell structure, as evidenced further by fast Fourier transform images. Through all studies, it is shown that the structure developed during mesophase pitch spinning altered the microstructure, thus impacting the mechanical properties, confirming a direct relationship between processing, structure, and properties.

Scherschel, Alexander [Univ. of Virginia, Charlott↗
DOE OSTI2024

AstroCLIP: a cross-modal foundation model for galaxies

ABSTRACT We present AstroCLIP, a single, versatile model that can embed both galaxy images and spectra into a shared, physically meaningful latent space. These embeddings can then be used – without any model fine-tuning – for a variety of downstream tasks including (1) accurate in-modality and cross-modality semantic similarity search, (2) photometric redshift estimation, (3) galaxy property estimation from both images and spectra, and (4) morphology classification. Our approach to implementing AstroCLIP consists of two parts. First, we embed galaxy images and spectra separately by pre-training separate transformer-based image and spectrum encoders in self-supervised settings. We then align the encoders using a contrastive loss. We apply our method to spectra from the Dark Energy Spectroscopic Instrument and images from its corresponding Legacy Imaging Survey. Overall, we find remarkable performance on all downstream tasks, even relative to supervised baselines. For example, for a task like photometric redshift prediction, we find similar performance to a specifically trained ResNet18, and for additional tasks like physical property estimation (stellar mass, age, metallicity, and specific-star-formation rate), we beat this supervised baseline by 19 per cent in terms of R2. We also compare our results with a state-of-the-art self-supervised single-modal model for galaxy images, and find that our approach outperforms this benchmark by roughly a factor of two on photometric redshift estimation and physical property prediction in terms of R2, while remaining roughly in-line in terms of morphology classification. Ultimately, our approach represents the first cross-modal self-supervised model for galaxies, and the first self-supervised transformer-based architectures for galaxy images and spectra.

Parker, Liam (ORCID:0009000749521674)↗
DOE OSTI2025

A bi-channel aided stitching of atomic force microscopy images

Microscopy is an essential tool in scientific research, enabling the visualization of structures at micro- and nanoscale resolutions. However, the field of microscopy often encounters limitations in field-of-view (FOV), restricting the amount of sample that can be imaged in a single capture. To overcome this limitation, image stitching techniques have been developed to seamlessly merge multiple overlapping images into a single, high-resolution composite. The images collected from microscope need to be optimally stitched before accurate physical information can be extracted from post analysis. However, the existing stitching tools either struggle to stitch images together when the microscopy images are feature sparse or cannot address all the transformations of images when performing image stitching. To address these issues, we propose a bi-channel aided feature-based image stitching method and demonstrate its use on Atomic Force Microscopy (AFM) generated Pantoea sp. YR343 biofilm and PTO thin film sample images as experimental data. The topographical channel image of AFM data captures the morphological details of the sample, and a stitched topographical image is desired for researchers. We utilize the amplitude and phase channels of AFM data to maximize the matching features and to estimate the position of the original topographical images and show that the proposed bi-channel aided stitching method outperforms the traditional direct stitching approach in AFM topographical image stitching task. Here, we demonstrated the application on AFM, but similar approaches could be employed of optical microscopy with brightfield and fluorescence channels. We believe this proposed workflow can serve as a valuable augmentation strategy for microscopy image stitching tasks and will benefit the experimentalist to avoid erroneous analysis and discovery due to incorrect stitching.

Atomic force microscopy↗
DOE OSTI2024

Operando real-space imaging of a structural phase transformation in the high-voltage electrode LixNi0.5Mn1.5O4

Abstract Discontinuous solid-solid phase transformations play a pivotal role in determining the properties of rechargeable battery electrodes. By leveraging operando Bragg Coherent Diffractive Imaging (BCDI), we investigate the discontinuous phase transformation in Li x Ni 0.5 Mn 1.5 O 4 within an operational Li metal coin cell. Throughout Li-intercalation, we directly observe the nucleation and growth of the Li-rich phase within the initially charged Li-poor phase in a 500 nm particle. Supported by the microelasticity model, the operando imaging unveils an evolution from a curved coherent to a planar semi-coherent interface driven by dislocation dynamics. Our data indicates negligible kinetic limitations from interface propagation impacting the transformation kinetics, even at a discharge rate of C/2 (80 mA/g). This study highlights BCDI’s capability to decode complex operando diffraction data, offering exciting opportunities to study nanoscale phase transformations with various stimuli.

Sun, Yifei (ORCID:0000000295625120)↗
DOE OSTI2025

Pxl

The code consists of several scripts. The image is fetched from a predefined URL, downloaded, and is displayed in the virtual environment at a predefined location. The image is also dynamically scaled to fit in the virtual environment depending on its size. The image is sent to have each pixel's value written to a CSV file. While the CSV is being written, a dictionary is populated with values that map each pixel to a temperature reading. The image's height, width, position and references to its corners are sent to make a button that is the same size as the image and placed right behind the image in the virtual scene. The button defines behavior that should be triggered when the user pushes the image. Unity provides visual cues that alert the user when they are about to touch something in the virtual environment. The code holds references to the cues for the left and right hands. The button queries the cue tracker and receives the position of the active cue in the form of a 3D vector. The 3D vector is compared to the bottom left corner of the image to transform the world space coordinates of the cue to the pixel space of the image. The specific pixel is approximated relative to how far left and up the user touched from the bottom left corner of the image. The approximated pixel value, which is represented as a coordinate in 2D space, is used to retrieve the temperature value associated with the approximated pixel. A window of text is then updated to display the temperature reading.

. Otis, KrystianeS [Idaho National Laboratory (INL↗
DOE OSTI2024

Rotational Millimeter-Wave Shoe Scanner Using the Discrete Fourier Transform for Backprojection-Based Image Reconstruction

An active 3D microwave / millimeter-wave shoe scanner was previously developed at the Pacific Northwest National Laboratory (PNNL) using two linear arrays scanned over a rectilinear aperture. The radar system chirps a frequency sweep from 10-40 GHz. These frequencies allow imaging through optically opaque material such as leather, rubber, plastics, and other dielectrics. The system was designed to detect concealed items in the soles of shoes while allowing people to leave their shoes on through a security checkpoint. To shrink the footprint of the system, a new iteration of the design has been developed that scans the two linear arrays over a circular aperture. This new footprint opens the possibility of it being installed in the floor of a cylindrical millimeter-wave body scanner. The backprojection-based multilayer dielectric image reconstruction developed at PNNL can easily handle arbitrary spatial sampling, accommodating the new rotational shoe scanner design. Commonly, the fast Fourier transform (FFT) is used to efficiently compute the range response from the data collected by the system as a preprocessing step to the backprojection algorithm. It was found that converting to range using the discrete Fourier transform (DFT) directly has some advantages over the FFT. For example, nonlinear and non-uniform frequency sweeps can easily be compensated for during the computation of the DFT and only the range bins of interest need to be computed and their spacing can be chosen arbitrarily. Because the range conversion step of the image reconstruction is the fastest part of the process there is very little speed penalty for using the DFT over the FFT and it can even increase the speed of image reconstruction when the ranges of interest are fewer than the total span that is calculated in the FFT.

Millimeter-wave imaging, microwave imaging, shoe s↗
DOE OSTI2026

FTIR imaging identifies alterations in lung tissue structure and biochemical composition in human idiopathic pulmonary fibrosis

Idiopathic Pulmonary Fibrosis (IPF) is a chronic, progressive, and fatal lung disease characterized by damage to the epithelial tissue and a reduced ability of the alveoli to repair themselves. This impaired repair process leads to abnormal accumulation of extracellular matrix (ECM), resulting in scarring and stiffening of lung tissue. Fourier transform infrared imaging (FTIRI) is a promising technique for imaging the biochemical changes related to fibrotic changes in a label-free and non-destructive manner, which can be analyzed to mark the progression of IPF. In this study, FTIRI was used to image human lung tissue biopsies with IPF and control biopsies without disease. In-depth spectral analyses were performed to observe the biochemical changes in the tissue composition using FTIRI. The parameters that were analyzed included collagen structure, total lipid content, lipid chain length, and phospholipids. Results showed a significant increase in lipid content in IPF compared to control, where long chain lipids dominated and phospholipids were reduced. Minor changes in collagen structure were also observed in IPF, likely attributed to the excess formation of extracellular matrix in the disease. These findings indicate that FTIRI has the potential to be a promising diagnostic technique to understand the molecular changes during IPF, as analysis of infrared data can reveal detailed biochemical information regarding disease progression and provide spatial insights on the molecular changes across the IPF lung tissue.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Contaminant Investigation and Pre‐Processing Opportunities for Textile‐To‐Textile Recycling

Millions of metric tons of textiles are landfilled or incinerated each year in the United States, with less than 1% of textiles recycled into new clothing or fabrics. To counter this trend, a growing number of companies and researchers are exploring how a circular economy can be applied to support textile‐to‐textile recycling. A significant barrier they face comes down to quickly and efficiently extracting pure feedstock material from post‐consumer garments that feature a mix of natural and synthetic fibers. Textile recyclers prefer pure feedstocks, as working with mixed sources typically means lower throughput, higher risk of equipment failure, and diminished business margins. To facilitate a circular economy for textiles, methods, and technologies are needed that can efficiently separate out materials and contaminants from end‐of‐life textiles to increase the flow of pure feedstocks to recyclers. This paper summarizes findings from interviews with a cross section of textile recyclers and from a review of literature to define basic feedstock requirements. In addition to our qualitative research, we deconstruct a bale of post‐consumer textiles and analyze them using computer‐vision imaging, Fourier transform infrared spectroscopy (FTIR), and machine learning. The resulting data are used to set system‐level design inputs for an automated contaminant removal system to process post‐consumer clothing into appropriate feedstocks for recycling. To set the system's levels for automated real‐time near‐infrared analysis, we identify the minimum percentage of primary material that any single garment in a load of used clothing must contain for the average of the full output stream to meet the target purity levels of recyclers. Here, the envisioned automated system can also address undesirable trace materials that might contaminate the processed stream by using imaging cameras coupled with artificial intelligence to identify sections of clothing for de‐trimming. Proof‐of‐concept machine learning algorithms are evaluated to locate and identify trims or garment areas with hidden contaminant materials. Integrating these methods into automated textile cutting systems can provide a cost‐effective means for increasing feedstock purity from used clothing, which can advance circularity for textiles by helping recyclers to reach production volumes and quality targets that were not possible solely with manual dismantling operations.

Parsons, Ryan [Rochester Institute of Technology, ↗
DOE OSTI2025

Spectral Performance of Multilayer Amorphous Selenium and Selenium–Tellurium Photodetectors

The ability to robustly and with scalability detect single photons in the visible spectrum with wavelength resolution would transform many imaging applications. Theoretical studies propose an array of carbon nanotubes (CNTs) functionalized with semiconductor quantum dots (QDs) as a physical realization of such photon sensors. In this work, we report approaches to synthesize these CNT-QD nanostructures using DNA as a smart glue to connect CNTs to QDs.

36 MATERIALS SCIENCE↗
DOE OSTI2026

3D reconstruction and neural rendering for adversarial machine learning

While evasion attacks on computer vision systems have been widely studied, creating attacks that remain effective under significant changes in viewpoint continues to be challenging. Traditional approaches often rely on affine transformations of images, but these approaches degrade at larger perspective shifts and often produce unrealistic or ineffective perturbations. Recent methods use differentiable renderers to improve viewpoint robustness, but they typically depend on manually constructed 3D models. We introduce a semi-automated pipeline that generates physically printable and perspective-invariant adversarial patches using only a small set of 2D images. Our method integrates 3D reconstruction, neural rendering, adversarial patch optimization, and an object detection victim model into a unified workflow. We use 2D Gaussian Splatting for high fidelity mesh reconstruction and FlexPara for surface parameterization that produces texture maps suitable for patch editing. Together, these components form a fully differentiable pipeline in PyTorch3D that links texture modification to model outputs, enabling efficient optimization of patches that remain effective across many viewpoints. The complete process, from image capture to patch printing and physical evaluation, can be completed within a few hours. We demonstrate the effectiveness of the resulting patches through attacks on the YOLOv8 object detection model and discuss remaining challenges and opportunities for improving robustness and scalability.

Singhvi, Vivaan [ORNL] (ORCID:0009000586288221)↗
DOE OSTI2025

Topology-imprinting in nonlinear metasurfaces

Flat optical components, or metasurfaces, have transformed optical imaging, data storage, information processing, and biomedical applications by providing unprecedented control over light-matter interactions. These nano-engineered structures enable compact, multidimensional manipulation of light’s amplitude, phase, polarization, and wavefront, producing scalar and vector beams with unique properties such as orbital angular momentum and knotted topologies. This flexibility has potential applications in optical communication and imaging, particularly in complex environments such as atmospheric turbulence and undersea scattering. However, designing metasurfaces for shorter wavelengths, such as visible and ultraviolet light, remains challenging due to fabrication limitations and material absorption. Here, we introduce an innovative concept called topology imprinting using judiciously designed all-dielectric nonlinear optical metasurfaces to replicate desired waveforms at fundamental and harmonic frequencies, opening promising avenues for advanced photonic applications.

42 ENGINEERING↗