SEARCH · Search NASA
Results for “object recognition”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Leveraging Prior Concept Learning Improves Generalization From Few Examples in Computational Models of Human Object Recognition
Humans quickly and accurately learn new visual concepts from sparse data, sometimes just a single example. The impressive performance of artificial neural networks which hierarchically pool afferents across scales and positions suggests that the hierarchical organization of the human visual system is critical to its accuracy. These approaches, however, require magnitudes of order more examples than human learners. We used a benchmark deep learning model to show that the hierarchy can also be leveraged to vastly improve the speed of learning. We specifically show how previously learned but broadly tuned conceptual representations can be used to learn visual concepts from as few as two positive examples; reusing visual representations from earlier in the visual hierarchy, as in prior approaches, requires significantly more examples to perform comparably. These results suggest techniques for learning even more efficiently and provide a biologically plausible way to learn new visual concepts from few examples.
Combined Effects of Three High-Energy Charged Particle Beams Important for Space Flight on Brain, Behavioral and Cognitive Endpoints in B6D2F1 Female and Male Mice
The radiation environment in deep space includes the galactic cosmic radiation with different proportions of all naturally occurring ions from protons to uranium. Most experimental animal studies for assessing the biological effects of charged particles have involved acute dose delivery for single ions and/or fractionated exposure protocols. Here, we assessed the behavioral and cognitive performance of female and male C 57 BL/6J × DBA2/J F1 (B6D2F1) mice 2 months following rapidly delivered, sequential irradiation with protons (1 GeV, 60%), 16 O (250 MeV/n, 20%), and 28 Si (263 MeV/n, 20%) at 0, 25, 50, or 200 cGy at 4-6 months of age. Cortical BDNF, CD68, and MAP-2 levels were analyzed 3 months after irradiation or sham irradiation. During the dark period, male mice irradiated with 50 cGy showed higher activity levels in the home cage than sham-irradiated mice. Mice irradiated with 50 cGy also showed increased depressive behavior in the forced swim test. When cognitive performance was assessed, sham-irradiated mice of both sexes and mice irradiated with 25 cGy showed normal responses to object recognition and novel object exploration. However, object recognition was impaired in female and male mice irradiated with 50 or 200 cGy. For cortical levels of the neurotrophic factor BDNF and the marker of microglial activation CD68, there were sex × radiation interactions. In females, but not males, there were increased CD68 levels following irradiation. In males, but not females, there were reduced BDNF levels following irradiation. A significant positive correlation between BDNF and CD68 levels was observed, suggesting a role for activated microglia in the alterations in BDNF levels. Finally, sequential beam irradiation impacted the diversity and composition of the gut microbiome. These included dose-dependent impacts and alterations to the relative abundance of several gut genera, such as Butyricicoccus and Lachnospiraceae . Thus, exposure to rapidly delivered sequential proton, 16 O ion, and 28 Si ion irradiation significantly affects behavioral and cognitive performance, cortical levels of CD68 and BDNF in a sex-dependent fashion, and the gut microbiome.
Endogenous amyloid-β mediates memory forgetting in the normal brain
Highlights: • Injection of anti-Aβ antibody into the ventricles prolonged object recognition memory. • It was also effective when injected after memory consolidation. • Genetic deletion of FcγRIIb, an Aβ receptor, prolonged object recognition memory. Amyloid beta (Aβ) is known to be one of the strong candidate molecules for initiating Alzheimer's disease and has been extensively studied in the light of disease pathophysiology. However, it is still elusive what roles Aβ play in the normal brain. In this study, we report that Aβ is required for memory forgetting in the normal brain. We monitored object recognition memory, and in order to quench soluble Aβ, we microinjected anti-Aβ antibody (4G8) into the ventricles after memory acquisition. Microinjection of anti-Aβ antibody prolonged the maintenance of object recognition memory. This effect appeared not to be due to modulation of memory consolidation since antibody injection after memory consolidation still had a similar effect on memory maintenance. Furthermore, the maintenance of object recognition memory was prolonged in Fcgr2b KO mice, which lacks IgG Fcγ receptor II-b (FcγRIIb), a receptor for soluble Aβ oligomers. Taken together, these findings suggest that endogenous Aβ is involved in memory forgetting in the normal brain.
Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition
Deep neural networks have surpassed human performance in key visual challenges such as object recognition, but require a large amount of energy, computation, and memory. In contrast, spiking neural networks (SNNs) have the potential to improve both the efficiency and biological plausibility of object recognition systems. Here we present a SNN model that uses spike-latency coding and winner-take-all inhibition (WTA-I) to efficiently represent visual stimuli using multi-scale parallel processing. Mimicking neuronal response properties in early visual cortex, images were preprocessed with three different spatial frequency (SF) channels, before they were fed to a layer of spiking neurons whose synaptic weights were updated using spike-timing-dependent-plasticity. We investigate how the quality of the represented objects changes under different SF bands and WTA-I schemes. We demonstrate that a network of 200 spiking neurons tuned to three SFs can efficiently represent objects with as little as 15 spikes per neuron. Furthermore, studying how core object recognition may be implemented using biologically plausible learning rules in SNNs may not only further our understanding of the brain, but also lead to novel and efficient artificial vision systems.
Effects of Six Sequential Charged Particle Beams on Behavioral and Cognitive Performance in B6D2F1 Female and Male Mice
The radiation environment astronauts are exposed to in deep space includes galactic cosmic radiation (GCR) with different proportions of all naturally occurring ions. To assist NASA with assessment of risk to the brain following exposure to a mixture of ions broadly representative of the GCR, we assessed the behavioral and cognitive performance of female and male C57BL/6J × DBA2/J F1 (B6D2F1) mice two months following rapidly delivered, sequential 6 beam irradiation with protons (1 GeV, LET = 0.24 keV, 50%), 4 He ions (250 MeV/n, LET = 1.6 keV/μm, 20%), 16 O ions (250 MeV/n, LET = 25 keV/μm 7.5%), 28 Si ions (263 MeV/n, LET = 78 keV/μm, 7.5%), 48Ti ions (1 GeV/n, LET = 107 keV/μm, 7.5%), and 56 Fe ions (1 GeV/n, LET = 151 keV/μm, 7.5%) at 0, 25, 50, or 200 cGy) at 4–6 months of age. When the activity over 3 days of open field habituation was analyzed in female mice, those irradiated with 50 cGy moved less and spent less time in the center than sham-irradiated mice. Sham-irradiated female mice and those irradiated with 25 cGy showed object recognition. However, female mice exposed to 50 or 200 cGy did not show object recognition. When fear memory was assessed in passive avoidance tests, sham-irradiated mice and mice irradiated with 25 cGy showed memory retention while mice exposed to 50 or 200 cGy did not. The effects of radiation passive avoidance memory retention were not sex-dependent. There was no effect of radiation on depressive-like behavior in the forced swim test. There was a trend toward an effect of radiation on BDNF levels in the cortex of males, but not for females, with higher levels in male mice irradiated with 50 cGy than sham-irradiated. Finally, sequential 6-ion irradiation impacted the composition of the gut microbiome in a sex-dependent fashion. Taxa were uncovered whose relative abundance in the gut was associated with the radiation dose received. Thus, exposure to sequential six-beam irradiation significantly affects behavioral and cognitive performance and the gut microbiome.
Semi-automatic image annotation using 3D LiDAR projections and depth camera data
Efficient image annotation is necessary to utilize deep learning object recognition neural networks in nuclear safeguards, such as for the detection and localization of target objects like nuclear material containers (NMCs). This capability can help automate the inventory accounting of different types of NMCs within nuclear storage facilities. The conventional manual annotation process is labor-intensive and time-consuming, hindering the rapid deployment of deep learning models for NMC identifications. This paper introduces a novel semi-automatic method for annotating 2D images of nuclear material containers (NMCs) by combining 3D light detection and ranging (LiDAR) data with color and depth camera images collected from a handheld scan system. The annotation pipeline involves an operator manually marking new target objects on a LiDAR-generated map, and projecting these 3D locations to images, thereby automatically creating annotations from the projections. The semi-automatic approach significantly reduces manual efforts and the expertise in image annotation that is required to perform the task, allowing deep learning models to be trained on-site within a few hours. The paper compares the performance of models trained on datasets annotated through various methods, including semi-automatic, manual, and commercial annotation services. The evaluation demonstrates that the semi-automatic annotation method achieves comparable or superior results, with a mean average precision (mAP) above 0.9, showcasing its efficiency in training object recognition models. Additionally, the paper explores the application of the proposed method to instance segmentation, achieving promising results in detecting multiple types of NMCs in various formations.
Automatic anatomy recognition in whole-body PET/CT images
Purpose: Whole-body positron emission tomography/computed tomography (PET/CT) has become a standard method of imaging patients with various disease conditions, especially cancer. Body-wide accurate quantification of disease burden in PET/CT images is important for characterizing lesions, staging disease, prognosticating patient outcome, planning treatment, and evaluating disease response to therapeutic interventions. However, body-wide anatomy recognition in PET/CT is a critical first step for accurately and automatically quantifying disease body-wide, body-region-wise, and organwise. This latter process, however, has remained a challenge due to the lower quality of the anatomic information portrayed in the CT component of this imaging modality and the paucity of anatomic details in the PET component. In this paper, the authors demonstrate the adaptation of a recently developed automatic anatomy recognition (AAR) methodology [Udupa et al., “Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images,” Med. Image Anal. 18, 752–771 (2014)] to PET/CT images. Their goal was to test what level of object localization accuracy can be achieved on PET/CT compared to that achieved on diagnostic CT images. Methods: The authors advance the AAR approach in this work in three fronts: (i) from body-region-wise treatment in the work of Udupa et al. to whole body; (ii) from the use of image intensity in optimal object recognition in the work of Udupa et al. to intensity plus object-specific texture properties, and (iii) from the intramodality model-building-recognition strategy to the intermodality approach. The whole-body approach allows consideration of relationships among objects in different body regions, which was previously not possible. Consideration of object texture allows generalizing the previous optimal threshold-based fuzzy model recognition method from intensity images to any derived fuzzy membership image, and in the process, to bring performance to the level achieved on diagnostic CT and MR images in body-region-wise approaches. The intermodality approach fosters the use of already existing fuzzy models, previously created from diagnostic CT images, on PET/CT and other derived images, thus truly separating the modality-independent object assembly anatomy from modality-specific tissue property portrayal in the image. Results: Key ways of combining the above three basic ideas lead them to 15 different strategies for recognizing objects in PET/CT images. Utilizing 50 diagnostic CT image data sets from the thoracic and abdominal body regions and 16 whole-body PET/CT image data sets, the authors compare the recognition performance among these 15 strategies on 18 objects from the thorax, abdomen, and pelvis in object localization error and size estimation error. Particularly on texture membership images, object localization is within three voxels on whole-body low-dose CT images and 2 voxels on body-region-wise low-dose images of known true locations. Surprisingly, even on direct body-region-wise PET images, localization error within 3 voxels seems possible. Conclusions: The previous body-region-wise approach can be extended to whole-body torso with similar object localization performance. Combined use of image texture and intensity property yields the best object localization accuracy. In both body-region-wise and whole-body approaches, recognition performance on low-dose CT images reaches levels previously achieved on diagnostic CT images. The best object recognition strategy varies among objects; the proposed framework however allows employing a strategy that is optimal for each object.
Chapter 18: Learning and Tracking Ad Hoc Fiducial Markers in Spatial Augmented Reality
We describe a spatial augmented reality system with a tangible user interface used to control computer simulations of complex systems. In spatial augmented reality, the user’s physical space is augmented with projected imagery, blending real objects with projected information, and a tangible user interface enables users to manipulate physical objects as controllers for interactive visualizations. Our system learns ad hoc objects in the user’s environment as fiducial markers (i.e., objects that are visually recognized and tracked). When combined with simulation and visualization tools, these interfaces allow the user to control simulations or ensembles of simulations via physical objects using apt metaphors. While other research has leveraged the use of depth cameras, our system enables the use of standard cameras in readily available smartphones and webcams and has an implementation that runs completely in JavaScript in the web browser. We discuss the prerequisite object-recognition requirements for such tangible user interfaces and describe computer-vision and machine-learning algorithms meeting those requirements. We conclude by presenting example applications, which are also available online.
Developmental exposure to corn grown on Lake Erie dredged material: a preliminary analysis
While corn is considered to be a healthy food option, common agricultural practices, such as the application of soil amendments, might be introducing contaminants of concern (COC) into corn plants. The use of dredged material, which contain contaminants such as heavy metals, polychlorinated biphenyls (PCBs) and polycyclic aromatic hydrocarbons (PAHs), as a soil amendment is increasing. Contaminants from these amendments can accumulate in corn kernels harvested from plants grown on these sediments and potentially biomagnify in organisms that consume them. The extent to which secondary exposure to such contaminants in corn affect the mammalian central nervous system has been virtually unexplored. In this preliminary study, we examine the effects of exposure to corn grown in dredge amended soil or a commercially available feed corn on behavior and hippocampal volume in male and female rats. Perinatal exposure to dredge-amended corn altered behavior in the open-field and object recognition tasks in adulthood. Additionally, dredge-amended corn led to a reduction in hippocampal volume in male but not female adult rats. These results suggest the need for future studies examining how dredge-amended crops and/or commercially available feed corn may be exposing animals to COC that can alter neurodevelopment in a sex-specific manner. This future work will provide insight into the potential long-term consequences of soil amendment practices on the brain and behavior.
SU-D-201-05: On the Automatic Recognition of Patient Safety Hazards in a Radiotherapy Setup Using a Novel 3D Camera System and a Deep Learning Framework
Purpose: Patient safety hazards such as a wrong patient/site getting treated can lead to catastrophic results. The purpose of this project is to automatically detect potential patient safety hazards during the radiotherapy setup and alert the therapist before the treatment is initiated. Methods: We employed a set of co-located and co-registered 3D cameras placed inside the treatment room. Each camera provided a point-cloud of fraxels (fragment pixels with 3D depth information). Each of the cameras were calibrated using a custom-built calibration target to provide 3D information with less than 2 mm error in the 500 mm neighborhood around the isocenter. To identify potential patient safety hazards, the treatment room components and the patient’s body needed to be identified and tracked in real-time. For feature recognition purposes, we used a graph-cut based feature recognition with principal component analysis (PCA) based feature-to-object correlation to segment the objects in real-time. Changes in the object’s position were tracked using the CamShift algorithm. The 3D object information was then stored for each classified object (e.g. gantry, couch). A deep learning framework was then used to analyze all the classified objects in both 2D and 3D and was then used to fine-tune a convolutional network for object recognition. The number of network layers were optimized to identify the tracked objects with >95% accuracy. Results: Our systematic analyses showed that, the system was effectively able to recognize wrong patient setups and wrong patient accessories. The combined usage of 2D camera information (color + depth) enabled a topology-preserving approach to verify patient safety hazards in an automatic manner and even in scenarios where the depth information is partially available. Conclusion: By utilizing the 3D cameras inside the treatment room and a deep learning based image classification, potential patient safety hazards can be effectively avoided.
Scaling the training of particle classification on simulated MicroBooNE events to multiple GPUs
Measurements in Liquid Argon Time Projection Chamber (LArTPC) neutrino detectors, such as the MicroBooNE detector at Fermilab, feature large, high fidelity event images. Deep learning techniques have been extremely successful in classification tasks of photographs, but their application to LArTPC event images is challenging, due to the large size of the events. Events in these detectors are typically two orders of magnitude larger than images found in classical challenges, like recognition of handwritten digits contained in the MNIST database or object recognition in the ImageNet database. Ideally, training would occur on many instances of the entire event data, instead of many instances of cropped regions of interest from the event data. However, such efforts lead to extremely long training cycles, which slow down the exploration of new network architectures and hyperparameter scans to improve the classification performance. We present studies of scaling a LArTPC classification problem on multiple architectures, spanning multiple nodes. The studies are carried out on simulated events in the MicroBooNE detector. We emphasize that it is beyond the scope of this study to optimize networks or extract the physics from any results here. Institutional computing at Pacific Northwest National Laboratory and the SummitDev machine at Oak Ridge National Laboratory’s Leadership Computing Facility have been used. To our knowledge, this is the first use of state-of-the-art Convolutional Neural Networks for particle physics and their attendant compute techniques onto the DOE Leadership Class Facilities. We expect benefits to accrue particularly to the Deep Underground Neutrino Experiment (DUNE) LArTPC program, the flagship US High Energy Physics (HEP) program for the coming decades.
Vision-Guided Automation and Assistance
Robotic automation and assistance is an excellent way to increase efficiency and safety when working in a potentially hazardous workspace. However, many traditional automation approaches rely on the precise placement of items to be manipulated. The scope of such automated tasks is also limited to the manipulation of one type of object since there is no way to distinguish between different objects. The addition of a vision pipeline, where robot vision is used to localize and recognize objects, can allow the robot to perform more complex tasks in a variable workspace. In my vision pipeline, a depth camera is used to generate a 3D point cloud of the workspace. Objects to be manipulated are isolated in the point cloud by filtering out shared surfaces and using Euclidean cluster extraction to group clusters of points. The locations of the objects are then estimated from the point cloud of the isolated objects. The localization of objects with robot vision allows for variable workspaces, reducing the overhead associated with requiring all objects to be precisely placed in the workspace. For object recognition, deep learning is used to train a neural network that classifies objects from the raw point clouds. More complex tasks that require multiple objects to be manipulated differently can be achieved through the vision pipeline. Through its ability to visualize the workspace, robot vision greatly improves efficiency and safety of automated and assistive tasks.
Machine Vision-based Robot Manipulators for Nuclear Applications - 20370
Decommissioning and dismantling of nuclear facilities are major challenges facing the nuclear industry. Robot manipulators with capabilities of restoring nuclear structures and significantly prolonging nuclear power generation in addition to their applications in decommissioning and dismantling (D and D) would immensely ease the above-mentioned challenge. Ability to remotely sense/monitor and analyze dangerous and hazardous environments such as nuclear reactors as well as performing reparatory tasks in such environments using robot manipulators require additional information from vision sensors. Structural sample collection and inspection as well as restoration require object and position information. Computer vision is used to obtain precise 3D position information. The current work uses the Sawyer robot with the integrated Cognex camera. The research presents an approach to facilitate and improve operations in a nuclear reactor using robotic vision control. The vision control system is modeled with Augmented Image Space-based visual servoing approach. Results showing accurate robot control. Object recognition is used to recognize the image and calculate it poses. To transform coordinates of the object's pose from the camera frame to the robotic frame transformation matrices are employed. Future work will employ a 3D camera (with depth information) using Denso robot. (authors)
Immersive Digital Twin Laboratory for Engineering Education (CRADA Final Report)
This project aimed to create an immersive digital twin laboratory that incorporates advanced tracking and visualization capabilities. In collaboration with Fort Lewis College, NREL designed a state-of-the-art physical visualization laboratory, developed a software platform to enable interaction with tracked physical objects in the laboratory, and provided proof-of-concept curricula that included manipulating these tracked objects. The project was initiated to address the growing need for innovative educational tools in engineering education. As renewable energy systems, particularly solar installations, become more complex, there is a pressing need to bridge the gap between theoretical knowledge and practical application. Traditional methods of teaching solar engineering concepts often fall short of providing students with a comprehensive, hands-on understanding. This immersive digital twin laboratory was conceived to fill that gap by creating a safe, non-energized setting where students can interact with augmented solar installation objects, gaining valuable insights into system performance, design, and maintenance. The project utilized extended reality (XR) technologies, including head-mounted displays (HMDs) and a whole-room optical motion tracking system, to connect physical objects with their digital twins in real time. The laboratory was equipped with MagicLeap 2 HMDs, supported by a Vicon Vero 2.2 Optical Tracking System, which provided precise 6-degrees-of-freedom (6-DOF) tracking. We developed a software platform to manage the interaction between the tracked physical objects and their virtual counterparts, enabling real-time data synchronization, object recognition, and virtual overlays. We designed the system to be flexible and extendable, allowing for future integration of additional objects and curriculum. This research advances the field of engineering education by demonstrating the potential of immersive digital twin environments. The laboratory provides a dynamic learning space where students can experiment, collaborate, and learn without the risks associated with live experimentation. The ability to simulate and manipulate solar installation objects under various conditions has broad implications for workforce development, particularly in renewable energy. The project also highlights the economic feasibility of using XR technologies in educational settings, offering a cost-effective solution for institutions looking to enhance their curriculum. By fostering a deeper understanding of solar energy systems, this work contributes to the broader goal of supporting the global energy transition and preparing the next generation of engineers and technicians.