SEARCH · Search NASA
Results for “Object Recognition”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Leveraging Prior Concept Learning Improves Generalization From Few Examples in Computational Models of Human Object Recognition
Humans quickly and accurately learn new visual concepts from sparse data, sometimes just a single example. The impressive performance of artificial neural networks which hierarchically pool afferents across scales and positions suggests that the hierarchical organization of the human visual system is critical to its accuracy. These approaches, however, require magnitudes of order more examples than human learners. We used a benchmark deep learning model to show that the hierarchy can also be leveraged to vastly improve the speed of learning. We specifically show how previously learned but broadly tuned conceptual representations can be used to learn visual concepts from as few as two positive examples; reusing visual representations from earlier in the visual hierarchy, as in prior approaches, requires significantly more examples to perform comparably. These results suggest techniques for learning even more efficiently and provide a biologically plausible way to learn new visual concepts from few examples.
Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition
Deep neural networks have surpassed human performance in key visual challenges such as object recognition, but require a large amount of energy, computation, and memory. In contrast, spiking neural networks (SNNs) have the potential to improve both the efficiency and biological plausibility of object recognition systems. Here we present a SNN model that uses spike-latency coding and winner-take-all inhibition (WTA-I) to efficiently represent visual stimuli using multi-scale parallel processing. Mimicking neuronal response properties in early visual cortex, images were preprocessed with three different spatial frequency (SF) channels, before they were fed to a layer of spiking neurons whose synaptic weights were updated using spike-timing-dependent-plasticity. We investigate how the quality of the represented objects changes under different SF bands and WTA-I schemes. We demonstrate that a network of 200 spiking neurons tuned to three SFs can efficiently represent objects with as little as 15 spikes per neuron. Furthermore, studying how core object recognition may be implemented using biologically plausible learning rules in SNNs may not only further our understanding of the brain, but also lead to novel and efficient artificial vision systems.
Effects of Six Sequential Charged Particle Beams on Behavioral and Cognitive Performance in B6D2F1 Female and Male Mice
The radiation environment astronauts are exposed to in deep space includes galactic cosmic radiation (GCR) with different proportions of all naturally occurring ions. To assist NASA with assessment of risk to the brain following exposure to a mixture of ions broadly representative of the GCR, we assessed the behavioral and cognitive performance of female and male C57BL/6J × DBA2/J F1 (B6D2F1) mice two months following rapidly delivered, sequential 6 beam irradiation with protons (1 GeV, LET = 0.24 keV, 50%), 4 He ions (250 MeV/n, LET = 1.6 keV/μm, 20%), 16 O ions (250 MeV/n, LET = 25 keV/μm 7.5%), 28 Si ions (263 MeV/n, LET = 78 keV/μm, 7.5%), 48Ti ions (1 GeV/n, LET = 107 keV/μm, 7.5%), and 56 Fe ions (1 GeV/n, LET = 151 keV/μm, 7.5%) at 0, 25, 50, or 200 cGy) at 4–6 months of age. When the activity over 3 days of open field habituation was analyzed in female mice, those irradiated with 50 cGy moved less and spent less time in the center than sham-irradiated mice. Sham-irradiated female mice and those irradiated with 25 cGy showed object recognition. However, female mice exposed to 50 or 200 cGy did not show object recognition. When fear memory was assessed in passive avoidance tests, sham-irradiated mice and mice irradiated with 25 cGy showed memory retention while mice exposed to 50 or 200 cGy did not. The effects of radiation passive avoidance memory retention were not sex-dependent. There was no effect of radiation on depressive-like behavior in the forced swim test. There was a trend toward an effect of radiation on BDNF levels in the cortex of males, but not for females, with higher levels in male mice irradiated with 50 cGy than sham-irradiated. Finally, sequential 6-ion irradiation impacted the composition of the gut microbiome in a sex-dependent fashion. Taxa were uncovered whose relative abundance in the gut was associated with the radiation dose received. Thus, exposure to sequential six-beam irradiation significantly affects behavioral and cognitive performance and the gut microbiome.
Semi-automatic image annotation using 3D LiDAR projections and depth camera data
Efficient image annotation is necessary to utilize deep learning object recognition neural networks in nuclear safeguards, such as for the detection and localization of target objects like nuclear material containers (NMCs). This capability can help automate the inventory accounting of different types of NMCs within nuclear storage facilities. The conventional manual annotation process is labor-intensive and time-consuming, hindering the rapid deployment of deep learning models for NMC identifications. This paper introduces a novel semi-automatic method for annotating 2D images of nuclear material containers (NMCs) by combining 3D light detection and ranging (LiDAR) data with color and depth camera images collected from a handheld scan system. The annotation pipeline involves an operator manually marking new target objects on a LiDAR-generated map, and projecting these 3D locations to images, thereby automatically creating annotations from the projections. The semi-automatic approach significantly reduces manual efforts and the expertise in image annotation that is required to perform the task, allowing deep learning models to be trained on-site within a few hours. The paper compares the performance of models trained on datasets annotated through various methods, including semi-automatic, manual, and commercial annotation services. The evaluation demonstrates that the semi-automatic annotation method achieves comparable or superior results, with a mean average precision (mAP) above 0.9, showcasing its efficiency in training object recognition models. Additionally, the paper explores the application of the proposed method to instance segmentation, achieving promising results in detecting multiple types of NMCs in various formations.
Chapter 18: Learning and Tracking Ad Hoc Fiducial Markers in Spatial Augmented Reality
We describe a spatial augmented reality system with a tangible user interface used to control computer simulations of complex systems. In spatial augmented reality, the user’s physical space is augmented with projected imagery, blending real objects with projected information, and a tangible user interface enables users to manipulate physical objects as controllers for interactive visualizations. Our system learns ad hoc objects in the user’s environment as fiducial markers (i.e., objects that are visually recognized and tracked). When combined with simulation and visualization tools, these interfaces allow the user to control simulations or ensembles of simulations via physical objects using apt metaphors. While other research has leveraged the use of depth cameras, our system enables the use of standard cameras in readily available smartphones and webcams and has an implementation that runs completely in JavaScript in the web browser. We discuss the prerequisite object-recognition requirements for such tangible user interfaces and describe computer-vision and machine-learning algorithms meeting those requirements. We conclude by presenting example applications, which are also available online.
Developmental exposure to corn grown on Lake Erie dredged material: a preliminary analysis
While corn is considered to be a healthy food option, common agricultural practices, such as the application of soil amendments, might be introducing contaminants of concern (COC) into corn plants. The use of dredged material, which contain contaminants such as heavy metals, polychlorinated biphenyls (PCBs) and polycyclic aromatic hydrocarbons (PAHs), as a soil amendment is increasing. Contaminants from these amendments can accumulate in corn kernels harvested from plants grown on these sediments and potentially biomagnify in organisms that consume them. The extent to which secondary exposure to such contaminants in corn affect the mammalian central nervous system has been virtually unexplored. In this preliminary study, we examine the effects of exposure to corn grown in dredge amended soil or a commercially available feed corn on behavior and hippocampal volume in male and female rats. Perinatal exposure to dredge-amended corn altered behavior in the open-field and object recognition tasks in adulthood. Additionally, dredge-amended corn led to a reduction in hippocampal volume in male but not female adult rats. These results suggest the need for future studies examining how dredge-amended crops and/or commercially available feed corn may be exposing animals to COC that can alter neurodevelopment in a sex-specific manner. This future work will provide insight into the potential long-term consequences of soil amendment practices on the brain and behavior.
Scaling the training of particle classification on simulated MicroBooNE events to multiple GPUs
Measurements in Liquid Argon Time Projection Chamber (LArTPC) neutrino detectors, such as the MicroBooNE detector at Fermilab, feature large, high fidelity event images. Deep learning techniques have been extremely successful in classification tasks of photographs, but their application to LArTPC event images is challenging, due to the large size of the events. Events in these detectors are typically two orders of magnitude larger than images found in classical challenges, like recognition of handwritten digits contained in the MNIST database or object recognition in the ImageNet database. Ideally, training would occur on many instances of the entire event data, instead of many instances of cropped regions of interest from the event data. However, such efforts lead to extremely long training cycles, which slow down the exploration of new network architectures and hyperparameter scans to improve the classification performance. We present studies of scaling a LArTPC classification problem on multiple architectures, spanning multiple nodes. The studies are carried out on simulated events in the MicroBooNE detector. We emphasize that it is beyond the scope of this study to optimize networks or extract the physics from any results here. Institutional computing at Pacific Northwest National Laboratory and the SummitDev machine at Oak Ridge National Laboratory’s Leadership Computing Facility have been used. To our knowledge, this is the first use of state-of-the-art Convolutional Neural Networks for particle physics and their attendant compute techniques onto the DOE Leadership Class Facilities. We expect benefits to accrue particularly to the Deep Underground Neutrino Experiment (DUNE) LArTPC program, the flagship US High Energy Physics (HEP) program for the coming decades.
Vision-Guided Automation and Assistance
Robotic automation and assistance is an excellent way to increase efficiency and safety when working in a potentially hazardous workspace. However, many traditional automation approaches rely on the precise placement of items to be manipulated. The scope of such automated tasks is also limited to the manipulation of one type of object since there is no way to distinguish between different objects. The addition of a vision pipeline, where robot vision is used to localize and recognize objects, can allow the robot to perform more complex tasks in a variable workspace. In my vision pipeline, a depth camera is used to generate a 3D point cloud of the workspace. Objects to be manipulated are isolated in the point cloud by filtering out shared surfaces and using Euclidean cluster extraction to group clusters of points. The locations of the objects are then estimated from the point cloud of the isolated objects. The localization of objects with robot vision allows for variable workspaces, reducing the overhead associated with requiring all objects to be precisely placed in the workspace. For object recognition, deep learning is used to train a neural network that classifies objects from the raw point clouds. More complex tasks that require multiple objects to be manipulated differently can be achieved through the vision pipeline. Through its ability to visualize the workspace, robot vision greatly improves efficiency and safety of automated and assistive tasks.
Machine Vision-based Robot Manipulators for Nuclear Applications - 20370
Decommissioning and dismantling of nuclear facilities are major challenges facing the nuclear industry. Robot manipulators with capabilities of restoring nuclear structures and significantly prolonging nuclear power generation in addition to their applications in decommissioning and dismantling (D and D) would immensely ease the above-mentioned challenge. Ability to remotely sense/monitor and analyze dangerous and hazardous environments such as nuclear reactors as well as performing reparatory tasks in such environments using robot manipulators require additional information from vision sensors. Structural sample collection and inspection as well as restoration require object and position information. Computer vision is used to obtain precise 3D position information. The current work uses the Sawyer robot with the integrated Cognex camera. The research presents an approach to facilitate and improve operations in a nuclear reactor using robotic vision control. The vision control system is modeled with Augmented Image Space-based visual servoing approach. Results showing accurate robot control. Object recognition is used to recognize the image and calculate it poses. To transform coordinates of the object's pose from the camera frame to the robotic frame transformation matrices are employed. Future work will employ a 3D camera (with depth information) using Denso robot. (authors)
Immersive Digital Twin Laboratory for Engineering Education (CRADA Final Report)
This project aimed to create an immersive digital twin laboratory that incorporates advanced tracking and visualization capabilities. In collaboration with Fort Lewis College, NREL designed a state-of-the-art physical visualization laboratory, developed a software platform to enable interaction with tracked physical objects in the laboratory, and provided proof-of-concept curricula that included manipulating these tracked objects. The project was initiated to address the growing need for innovative educational tools in engineering education. As renewable energy systems, particularly solar installations, become more complex, there is a pressing need to bridge the gap between theoretical knowledge and practical application. Traditional methods of teaching solar engineering concepts often fall short of providing students with a comprehensive, hands-on understanding. This immersive digital twin laboratory was conceived to fill that gap by creating a safe, non-energized setting where students can interact with augmented solar installation objects, gaining valuable insights into system performance, design, and maintenance. The project utilized extended reality (XR) technologies, including head-mounted displays (HMDs) and a whole-room optical motion tracking system, to connect physical objects with their digital twins in real time. The laboratory was equipped with MagicLeap 2 HMDs, supported by a Vicon Vero 2.2 Optical Tracking System, which provided precise 6-degrees-of-freedom (6-DOF) tracking. We developed a software platform to manage the interaction between the tracked physical objects and their virtual counterparts, enabling real-time data synchronization, object recognition, and virtual overlays. We designed the system to be flexible and extendable, allowing for future integration of additional objects and curriculum. This research advances the field of engineering education by demonstrating the potential of immersive digital twin environments. The laboratory provides a dynamic learning space where students can experiment, collaborate, and learn without the risks associated with live experimentation. The ability to simulate and manipulate solar installation objects under various conditions has broad implications for workforce development, particularly in renewable energy. The project also highlights the economic feasibility of using XR technologies in educational settings, offering a cost-effective solution for institutions looking to enhance their curriculum. By fostering a deeper understanding of solar energy systems, this work contributes to the broader goal of supporting the global energy transition and preparing the next generation of engineers and technicians.
Object Detection and Recognition with PointPillars in LiDAR Point Clouds – Comparisions
In the field of autonomous systems, neural networks have been leveraged for object detection and recognition in 2-dimensional images captured by cameras. Other types of sensors are available for sensing surroundings, including LiDAR sensors, and corresponding networks have been developed to perform detection and recognition in the point clouds generated by these sensors. The approaches are similar, both perform convolutions, but have distinct characteristics and challenges. In designing and configuring autonomous systems, a variety of LiDAR sensors are available, along with configurable deep neural networks to leverage their data. This work presents a review of the PointPillars network, an evolution of the seminal PointNet, comparing accuracy and training time relative to different LiDAR sensors, network and training parameters, CPU and GPU hardware, and the criticality of the use of reflective intensity as a feature. The value of using reflectivity as a predictive feature is explored and quantified to determine if it makes a significant difference in accuracy of the PointPillars network. Two separate LiDAR sensors are utilized, a 16-plane and a 32-plane, and corresponding accuracies and training times with the PointPillars network are evaluated.
Deep Learning Systems for Increased Safeguards Surveillance Review Productivity
Nuclear safeguards inspectors expend significant time and maintain intense focus in reviewing video surveillance for safeguards relevant events. To increase efficiency and reduce the time burden of safeguards inspectors performing surveillance data review, this paper presents a novel deep learning (DL) systems concept to integrate generalized DL models into the safeguards surveillance review workflow. The Agency is investigating several DL algorithms for object recognition, localization, tracking, and flagging relevant activities. The project team is working closely with nuclear safeguards inspectors to identify review use cases (based on specific safeguards objectives) and collect their associated requirements. We focused on CANDU and LWR Nuclear Power Plants (NPPs) and their associated dry storage areas as these present a particularly heavy burden on the inspector surveillance review process due to the number of these facilities under safeguards worldwide. Initial DL algorithm results on safeguards data are promising. Using a convolutional neural network, the team attained a mean average precision (mAP) of 92.9% identifying and localizing spent fuel (SF) casks from a 475 surveillance image dataset. Further, the team had initial success in training a recurrent neural network to identify reactor area activities in video clips, successfully indicating when SF casks enter or exit a pool. We discuss how such DL algorithms would be integrated into the Next Generation Surveillance Review (NGSR) software application. Another issue impacting review productivity is the long time inspectors may have to wait when running these algorithms in NGSR. We present a concept to pre-process remotely collected surveillance data with DL models as the data arrives to IAEA headquarters so that results are already available when starting a new review in NGSR. The proposed DL system concept shows a pathway and workflow for increasing an inspector’s surveillance review productivity by quickly and accurately identifying declared and undeclared safeguards relevant objects and activities in large quantities of surveillance imagery data.
Effectiveness of Deep Learning Trained on SynthCity Data for Urban Point-Cloud Classification
3D object recognition is one of the most popular areas of study in computer vision. Many of the more recent algorithms focus on indoor point clouds, classifying 3D geometric objects, and segmenting outdoor 3D scenes. One of the challenges of the classification pipeline is finding adequate and accurate training data. Hence, this article seeks to evaluate the accuracy of a synthetically generated data set called SynthCity, tested on two mobile laser-scan data sets. Varying levels of noise were applied to the training data to reflect varying levels of noise in different scanners. The chosen deep-learning algorithm was Kernel Point Convolution, a convolutional neural network that uses kernel points in Euclidean space for convolution weights.
Photon-Sparse, Poisson Light-Sheet Microscopy
Light-sheet microscopy has revolutionized bioimaging by enabling approximately an order of magnitude reduction in specimen irradiance compared to confocal imaging. Here, we introduce a light-sheet imaging system that enables an additional order of magnitude reduction in specimen irradiance by operating at the Poisson limit. To operate at this limit, we integrated classical illumination with single-photon detection and wavelet-based image reconstruction. This integration enabled brightness quantification and object recognition from fewer than one detected photon per image pixel, corresponding to more than 10-fold lower irradiance levels than modern systems. We demonstrate how such photon-sparse imaging can eradicate photobleaching and enable both dim and bright object imaging, thus, further enhancing the related gains of light-sheet microscopy.
Bringing Different Views Together: A Hybrid Cooperative Perception Framework for Connected Autonomous Vehicles
Cooperative perception will be essential for connected autonomous vehicles to enhance object recognition and optimize path planning by sending data information about the surrounding environment. However, an inherent challenge in existing systems is the high bandwidth cost of transmitting information in real-time, which restricts cooperative perception’s practicality. Here, this work presents a hybrid cooperative perception fusion framework aimed at mitigating this issue by optimizing data transmission according to available bandwidth or through data reduction techniques. Our methods ensure that vehicles can rapidly transmit high-confidence data without overwhelming the network. Experimental results indicate that our methodology substantially diminishes data transmission sizes while maintaining object detection accuracy. For cooperative perception in autonomous vehicle systems, our approach provides a scalable and effective way to get past the bandwidth barrier.
Non-DNA radiosensitive targets that initiate persistent behavioral deficits in rats exposed to space radiation
Predicting future CNS risks for astronauts during deep-space missions will rely substantially on ground-based rodent data with space-relevant ions and behaviors. For rats, the accumulated evidence indicates that less densely ionizing radiation, such as 4 He and 12 C ions, induce behavior deficits at lower doses than densely ionizing ions, such as 48 Ti and 56 Fe. However, this observation conflicts with standard somatic radiobiology, in which densely ionizing ions are generally more effective than less densely ionizing ions, and where the DNA/nucleus is the accepted target for radiation-induced tumorigenesis, cytogenetic aberrations, genetic mutations, and reproductive cell death. To gain deeper insight into the subcellular nature of the radiation targets for behavior risks, we compared the effects of dose, fluence, and linear energy transfer (LET) of 4 He and 56 Fe particles using existing datasets for four distinct behavioral outcomes in rats: elevated plus maze (EPM-anxiety), novel object recognition (NOR-memory), operant responding (OR-response to environmental stimuli), and attentional set-shifting (ATSET-cognitive flexibility). We confirmed that less densely ionizing particles (except protons) showed ~100-fold lower threshold doses than densely ionizing particles for behavioral deficits (0.1–1 cGy for 4 He vs. 15–100 cGy for 56 Fe). However, when analyzed by fluence the behavioral responses converged, indicating that 4 He and 56 Fe were equally effective on a per-track basis. When analyzed by LET, there were ~100-fold differences in the LET for maximum effectiveness for behavioral deficits and DNA endpoints (~1 vs ~100 keV/μm, respectively). These unique features of radiation-induced behavioral deficits (high sensitivity to particles in the 1-keV/μm range, insensitivity to protons in the 0.2 keV/μm range, and isofluence dependence for particles with LET>1 keV/μm) provide evidence in support of a new hypothesis of sub-micron sized radiosensitive targets for behavioral effects consistent with the thickness of plasma membranes and/or small subcellular structures, smaller than a whole synapse. Like our behavior findings, mouse immature oocyte killing which is known to have a plasma membrane target was also better explained by fluence, rather than dose. In contrast, fluence analyses for DNA/nuclear endpoints in somatic cells (e.g., tumor induction, chromosome aberrations) showed opposite results, suggesting that behavior targets are not DNA. Our findings raise questions regarding the identity of subcellular targets and the multi-cellular functional unit for behavior risks, low-dose susceptibility, and generalizability from rat to other species and astronauts.
Perovskite neural trees
Trees are used by animals, humans and machines to classify information and make decisions. Natural tree structures displayed by synapses of the brain involves potentiation and depression capable of branching and is essential for survival and learning. Demonstration of such features in synthetic matter is challenging due to the need to host a complex energy landscape capable of learning, memory and electrical interrogation. We report experimental realization of tree-like conductance states at room temperature in strongly correlated perovskite nickelates by modulating proton distribution under high speed electric pulses. This demonstration represents physical realization of ultrametric trees, a concept from number theory applied to the study of spin glasses in physics that inspired early neural network theory dating almost forty years ago. We apply the tree-like memory features in spiking neural networks to demonstrate high fidelity object recognition, and in future can open new directions for neuromorphic computing and artificial intelligence.