CMOS sum and difference imager with on-chip charge-leakage compensation
This paper presents a new technique for implementing a low-power CMOS imager with simultaneous on-chip computation of the difference and summ of two successive frames.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This paper presents a new technique for implementing a low-power CMOS imager with simultaneous on-chip computation of the difference and summ of two successive frames.
Alaska’s boreal forest is facing unprecedented challenges under rapid climate warming (increasingly severe fires, droughts, pest/disease outbreaks) that may destabilize its function as a global carbon sink. Forests near Fairbanks may be especially vulnerable, impacting air quality and ecosystem services. We combined GT (ground truthing) with Airborne Visible InfraRed Imaging Spectrometer (AVIRIS-NG) images collected by the NASA Arctic-Boreal Vulnerability Experiment (ABoVE) program (2017-2019) to assess landcover change at five recently burned sites (2001-2019) of different fire severities and moisture regimes within 30 miles of Fairbanks. GT included tree seedling counts, understory % cover and >50% leaf canopy color assessment. 36 circular plots (1/30 ha radius) including 6 moderate to severely burned plots were selected across sites. 31 additional sites including 12 burned sites were geotagged in photos. AVIRIS images were processed from 29 spectral bands selected to identify changes in chlorophyll and water content. Images were segmented into natural boundaries (polygons) using ENVI 5.5 software. A spectral library of 8 AVIRIS bands with high between-class/low within-class variation was used in two random forest models to predict vegetation classes (model 1: 12 classes, model 2: 14 classes) in each AVIRIS scene, using 20% of the data as training data. Model 2 classified 20% more polygons overall, but only 42% of GT/geotagged polygons were correctly classified by both models. More forest sites were correctly classified (63%) than open vegetation (32%) or post-fire sites (46%). 50% of aspen forest and post-fire polygons were misclassified as shrubland. GT revealed that post-fire plots supported 134,000 (± 48,000) tree seedlings and saplings ha-1 (0.2 - 4 m height, 64% deciduous) versus 2500 (± 2100) shrubs ha-1 (1-6 m height). > 50% canopy browning was observed in conifer forest (8 plots) with no signs of insect infestation. Canopy herbivory > 50% (leaf miner, leaf beetle) and moose herbivory of tree bark was seen across aspen sites. Our study suggests: 1) low canopy vegetation presents challenges for improved landcover classification, and 2) aspen forest should be differentiated in vegetation maps which would aid in tracking herbivory.
Deep learning has quickly become a necessity for selfdriving vehicles on Earth. In contrast, the self-driving vehicles on Mars, including NASA’s latest rover, Perseverance, which is planned to land on Mars in February 2021, are still driven by classical machine vision systems. Deep learning capabilities, such as semantic segmentation and object recognition, would substantially benefit the safety and productivity of ongoing and future missions to the red planet. To this end, we created the first large-scale dataset, AI4Mars, for training and validating terrain classification models for Mars, consisting of ~326K semantic segmentation full image labels on 35K images from Curiosity, Opportunity, and Spirit rovers, collected through crowdsourcing. Each image was labeled by ~10 people to ensure greater quality and agreement of the crowdsourced labels. It also includes ~1.5K validation labels annotated by the rover planners and scientists from NASA’s MSL (Mars Science Laboratory) mission, which operates the Curiosity rover, and MER (Mars Exploration Rovers) mission, which operated the Spirit and Opportunity rovers. We trained a DeepLabv3 model on the AI4Mars training dataset and achieved over 96% overall classification accuracy on the test set. The dataset is made publicly available.1
Woven, ablative thermal protection system (TPS) materials provide a robust option for aggressive (re)entries and thus have been baselined for the upcoming Mars Sample Return (MSR) mission's Earth Entry System (EES). The reliability requirements for MSR-EES necessitate understanding of material property variability, which could be significant given the complex structure and anisotropic nature of properties in TPS weaves, as well as the response to potential impact with micrometeoroids or orbital debris during the EES re-entry. The TPS Certification by Analysis effort within the Entry Systems Modeling project seeks to provide computational models and analyses that support the certification against such material-based risks. For the present talk, focus will be given to the characterization of baseline woven TPS material properties and mechanical failure limits, which entails (1) use of computational techniques (e.g., machine learning) to interpret computed tomography images of the weave to generate representative structural models and (2) application of multiscale material modeling approaches to characterize thermomechanical and failure properties.
We previously presented Soil Property and Object Classification (SPOC), a machine learning-based terrain classifier for Mars rovers, for automatically segmenting rover images by its surface type such as sand and bedrock. This paper presents a number of practical improvements to pave the way for potential future onboard deployment. First, we achieved 97.0% overall pixel accuracy, evaluated against the classification generated by human experts on images from Mars Science Laboratory (MSL) missions. The substantial increase in accuracy was primarily enabled by the sheer volume of data used for training; we created a new large-scale dataset of Martian terrain labels, namely AI4Mars, which contains more than 400k labels contributed by citizen scientists for 50k images taken by the Mars Exploration Rovers (MER) and Mars Science Laboratory (MSL) rover. Second, we demonstrated that SPOC can quickly adapt to a new mission landed on a previously unseen site. Specifically, we pretrained a model with MER and MSL data from the AI4Mars dataset and then adapted to the Mars 2020 Rover (M2020) by feeding a small volume of data between Sol 0 and 157; the adapted model was tested on Sol 200-203 and resulted in 84.2% overall pixel accuracy and 93.4% reliability (recall) for detecting sand, the most concerning class for rover’s traversability. Third, we found that pretraining can substantially mitigate the decline of accuracy over time. We showed that the performance of a SPOC model pretrained with the ImageNet dataset and then trained by MSL images only up to Sol 390 remains comparable to a model trained by images up to Sol 1689 on the test data after Sol 1689. Fourth, we reimplemented SPOC with a light-weight convolutional neural network (CNN), MobileNetV2, which typically runs within tens of milliseconds (ms) on mobile processors such as Qualcomm’s Snapdragon. Finally, we released the AI4Mars dataset to the public to encourage open innovation.
Explore the source record for details and available documents.
Plan to submit this to the Journal of Statistical Analysis and Data Mining for their special issue on CoDA (Conference on Data Analysis, a meeting of analysts and scientists from the national labs)
The paper presents the results from a multi-year effort to develop and validate image processing methods forselecting the best physical models based on solar image observations. The approach consists of selecting the physical models based on their agreement with coronal holes extracted from the images. Ultimately, the goal is to use physical models to predict geomagnetic storms. We decompose the problem into three subproblems: (i) coronal hole segmentation based on physical constraints, (ii) matching clusters of coronal holes between different maps, and (iii) physical map classification. For segmenting coronal holes, we develop a multi-modal method that uses segmentation maps from three different methods to initialize a level-set method that evolves the initial coronal hole segmentation to the magnetic boundary. Then, we introducea new method based on Linear Programming for matchingclusters of coronal holes. The final matching is then performedusing Random Forests. The methods were carefully validatedusing consensus maps derived from multiple readers, manualclustering, manual map classification, and method validation for50 maps. The proposed multi-modal segmentation method significantly outperformed SegNet, U-net, Henney-Harvey, and FCNby providing accurate boundary detection. Overall, the methodgave a 95.5% map classification accuracy.
Purpose: Dictionary learning based method has attracted more and more attentions in low-dose CT due to the superior performance on suppressing noise and preserving structural details. Considering the structures and noise vary from region to region in one imaging object, we propose a region-specific dictionary learning method to improve the low-dose CT reconstruction. Methods: A set of normal-dose images was used for dictionary learning. Segmentations were performed on these images, so that the training patch sets corresponding to different regions can be extracted out. After that, region-specific dictionaries were learned from these training sets. For the low-dose CT reconstruction, a conventional reconstruction, such as filtered back-projection (FBP), was performed firstly, and then segmentation was followed to segment the image into different regions. Sparsity constraints of each region based on its dictionary were used as regularization terms. The regularization parameters were selected adaptively according to different regions. A low-dose human thorax dataset was used to evaluate the proposed method. The single dictionary based method was performed for comparison. Results: Since the lung region is very different from the other part of thorax, two dictionaries corresponding to lung region and the rest part of thorax respectively were learned to better express the structural details and avoid artifacts. With only one dictionary some artifact appeared in the body region caused by the spot atoms corresponding to the structures in the lung region. And also some structure in the lung regions cannot be recovered well by only one dictionary. The quantitative indices of the result by the proposed method were also improved a little compared to the single dictionary based method. Conclusion: Region-specific dictionary can make the dictionary more adaptive to different region characteristics, which is much desirable for enhancing the performance of dictionary learning based method.
Crystalline materials exhibit long-range ordered lattice unit, within which resides nonperiodic structural features called defects. These crystallographic defects play a vital role in determining the physical and mechanical properties of a wide range of material systems. While computer vision has demonstrated success in recognizing feature patterns in images with well defined contrast, automated identification of nanometer scale crystallographic defects in electron micrographs governed by complex contrast mechanism is still a challenging task. Here, building upon an advanced defect imaging mode that offers high feature clarity, we introduce DefectSegNet - a new convolutional neural network (CNN) architecture that performs semantic segmentation of three common crystallographic defects in structural alloys: dislocation lines, precipitates and voids. Results from supervised training of a small set of high-quality defect images of steels show high pixel-wise accuracy across all three types of defects: 91.60 ± 1.77% on dislocations, 93.39 ± 1.00% on precipitates, and 98.85 ± 0.56% on voids. We discuss the sources of uncertainties in CNN prediction, and the training data in terms of feature density, representation and homogeneity and their effects on deep learning performance. Further defect quantification using DefectSegNet prediction outperforms human expert average, presenting a promising new workflow for fast and statistically meaningful quantification of materials defects.
Imaging nanoscale radiation-induced defects using the transmission electron microscope (TEM) is a key factor in the successful implementation of materials for nuclear energy structural applications. Analyzing each defect in a TEM micrograph is currently a manual task. To identify the defects in a single image can take anywhere from 15 min to an hour and a project can require the analysis of anywhere from tens to ≥ 100 images. Here, we use artificial intelligence (AI) models to automate this task. For simplification, we evaluated images with only a single type of defect; helium bubbles. Additionally, we performed semantic segmentation of these helium bubble defects in electron microscopy images of irradiated FeCrAl alloys using a deep learning DefectSegNet model. This model, which was previously used to classify crystal defects, is inspired by the classic DenseNet and U-Net image segmentation models. It claims high spatial resolution, but has poor performance at object boundaries. Our paper improves the DefectSegNet model’s application by adding two new features. First, the DefectSegNet model is applied not only to perform calculation pixel-wise but also object (or feature) wise. Because object-wise metrics are directly relevant to our final goal of detecting bubbles, whereas pixel-wise classification is only an intermediate step, it is an important part of our study. Second, a distance map loss (DML) function has been added to increase its performance at object boundaries. It is crucial to accurately represent defects boundaries, especially bubbles, in order to track the bubble-induced swelling caused by irradiation. The boundary-focused DML function is also compared to other loss functions like Cross-entropy, Weighted Binary Cross Entropy (WBCE), Dice and Intersection over Union (IOU). Finally, by incorporating new features, we found a marked improvement on segmentation quality and better shape preservation at the boundaries and areas of the bubbles.
We present advances with a 32 element scalable, segmented dual mode imager. Scaling up the number of cells results in a 1.4 increase in efficiency over a system we deployed last year. Variable plane separation has been incorporated which further improves the efficiency of the detector. By using 20 cm diameter cells we demonstrate that we could increase sensitivity by a factor of 6. We further demonstrate gamma ray imaging in from Compton scattering. This feature allows for powerful dual mode imaging. Selected results are presented that demonstrate these new capabilities.
We present advances with a 32 element scalable, segmented dual mode imager. Scaling up the number of cells results in a 1.4 increase in efficiency over a system we deployed last year. Variable plane separation has been incorporated which further improves the efficiency of the detector. By using 20 cm diameter cells we demonstrate that we could increase sensitivity by a factor of 6. We further demonstrate gamma ray imaging in from Compton scattering. This feature allows for powerful dual mode imaging. Selected results are presented that demonstrate these new capabilities.
Purpose: Proper delineation of ocular anatomy in 3-dimensional (3D) imaging is a big challenge, particularly when developing treatment plans for ocular diseases. Magnetic resonance imaging (MRI) is presently used in clinical practice for diagnosis confirmation and treatment planning for treatment of retinoblastoma in infants, where it serves as a source of information, complementary to the fundus or ultrasonographic imaging. Here we present a framework to fully automatically segment the eye anatomy for MRI based on 3D active shape models (ASM), and we validate the results and present a proof of concept to automatically segment pathological eyes. Methods and Materials: Manual and automatic segmentation were performed in 24 images of healthy children's eyes (3.29 ± 2.15 years of age). Imaging was performed using a 3-T MRI scanner. The ASM consists of the lens, the vitreous humor, the sclera, and the cornea. The model was fitted by first automatically detecting the position of the eye center, the lens, and the optic nerve, and then aligning the model and fitting it to the patient. We validated our segmentation method by using a leave-one-out cross-validation. The segmentation results were evaluated by measuring the overlap, using the Dice similarity coefficient (DSC) and the mean distance error. Results: We obtained a DSC of 94.90 ± 2.12% for the sclera and the cornea, 94.72 ± 1.89% for the vitreous humor, and 85.16 ± 4.91% for the lens. The mean distance error was 0.26 ± 0.09 mm. The entire process took 14 seconds on average per eye. Conclusion: We provide a reliable and accurate tool that enables clinicians to automatically segment the sclera, the cornea, the vitreous humor, and the lens, using MRI. We additionally present a proof of concept for fully automatically segmenting eye pathology. This tool reduces the time needed for eye shape delineation and thus can help clinicians when planning eye treatment and confirming the extent of the tumor.
Purpose: To compare interobserver variations in delineating the whole breast for treatment planning using two contouring methods. Methods and Materials: Autosegmented contours were generated by a deformable image registration-based breast segmentation method (DEF-SEG) by mapping the whole breast clinical target volume (CTVwb) from a template case to a new patient case. Eight breast radiation oncologists modified the autosegmented contours as necessary to achieve a clinically appropriate CTVwb and then recontoured the same case from scratch for comparison. The times to complete each approach, as well as the interobserver variations, were analyzed. The template case was also mapped to 10 breast cancer patients with a body mass index of 19.1-35.9 kg/m{sup 2}. The three-dimensional surface-to-surface distances and volume overlapping analyses were computed to quantify contour variations. Results: The median time to edit the DEF-SEG-generated CTVwb was 12.9 min (range, 3.4-35.9) compared with 18.6 min (range, 8.9-45.2) to contour the CTVwb from scratch (30% faster, p = 0.028). The mean surface-to-surface distance was noticeably reduced from 1.6 mm among the contours generated from scratch to 1.0 mm using the DEF-SEG method (p = 0.047). The deformed contours in 10 patients achieved 94% volume overlap before correction and required editing of 5% (range, 1-10%) of the contoured volume. Conclusion: Significant interobserver variations suggested a lack of consensus regarding the CTVwb, even among breast cancer specialists. Using the DEF-SEG method produced more consistent results and required less time. The DEF-SEG method can be successfully applied to patients with different body mass indexes.
Purpose: To propose and validate a novel and accurate deformable image registration (DIR) scheme to facilitate dose accumulation among treatment fractions of high-dose-rate (HDR) gynecological brachytherapy. Method: We have developed a method to adapt DIR algorithms to gynecologic anatomies with HDR applicators by incorporating a segmentation step and a point-matching step into an existing DIR framework. In the segmentation step, random walks algorithm is used to accurately segment and remove the applicator region (AR) in the HDR CT image. A semi-automatic seed point generation approach is developed to obtain the incremented foreground and background point sets to feed the random walks algorithm. In the subsequent point-matching step, a feature-based thin-plate spline-robust point matching (TPS-RPM) algorithm is employed for AR surface point matching. With the resulting mapping, a DVF characteristic of the deformation between the two AR surfaces is generated by B-spline approximation, which serves as the initial DVF for the following Demons DIR between the two AR-free HDR CT images. Finally, the calculated DVF via Demons combined with the initial one serve as the final DVF to map doses between HDR fractions. Results: The segmentation and registration accuracy are quantitatively assessed by nine clinical HDR cases from three gynecological cancer patients. The quantitative results as well as the visual inspection of the DIR indicate that our proposed method can suppress the interference of the applicator with the DIR algorithm, and accurately register HDR CT images as well as deform and add interfractional HDR doses. Conclusions: We have developed a novel and robust DIR scheme that can perform registration between HDR gynecological CT images and yield accurate registration results. This new DIR scheme has potential for accurate interfractional HDR dose accumulation. This work is supported in part by the National Natural ScienceFoundation of China (no 30970866 and no 81301940)
Purpose: The anatomical noise power spectra (NPS) for differential phase contrast (DPC) and dark field (DF) imaging have recently been characterized using a power-law model with two parameters, alpha and beta, an innovative extension to the methodology used in x-ray attenuation based breast imaging such as mammography, DBT, or cone-beam CT. Beta values of 3.6, 2.6, and 1.3 have been measured for absorption, DPC, and DF respectively for cadaver breasts imaged in the coronal plane; these dramatic differences should be reflected in their detection performance. The purpose of this study was to determine the impact of anatomical noise on breast calcification detection and compare the detection performance of the three contrast mechanisms of a multi-contrast x-ray imaging system. Methods: In our studies, a calcification image object was segmented out of the multi-contrast images of a cadaver breast specimen. 50 measured total NPS were measured from breast cadavers directly. The ideal model observer detectability was calculated for a range of doses (5–100%) and a range of calcification sizes (diameter = 0.25–2.5 mm). Results: Overall we found the highest average detectability corresponded to DPC imaging (7.4 for 1 mm calc.), with DF the next highest (3.8 for 1 mm calc.), and absorption the lowest (3.2 for 1 mm calc.). However, absorption imaging also showed the slowest dependence on dose of the three modalities due to the significant anatomical noise. DPC showed a peak detectability for calcifications ∼1.25 mm in diameter, DF showed a peak for calcifications around 0.75 mm in diameter, and absorption imaging had no such peak in the range explored. Conclusion: Understanding imaging performance for DPC and DF is critical to transition these modalities to the clinic. The results presented here offer new insight into how these modalities complement absorption imaging to maximize the likelihood of detecting early breast cancers. J. Garrett, Y. Ge, K. Li: Nothing to disclose. G.-H. Chen: Research funded, GE Healthcare; Research funded, Siemens AX.
Tattoo segmentation is challenging due to the complexity and large variance in tattoo structures. We have developed a segmentation algorithm for nding tattoos in an image. Our basic idea is split-merge: split each tattoo image into clusters through a bottom-up process, learn to merge the clusters containing skin and then distinguish tattoo from the other skin via top-down prior in the image itself. Tattoo segmentation with unknown number of clusters is transferred to a gure-ground segmentation. We have applied our segmentation algorithm on a tattoo dataset and the results have shown that our tattoo segmentation system is e cient and suitable for further tattoo classi cation and retrieval purpose.