SEARCH · Search NASA

Results for “VISUAL TASK”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2026

A Real2Sim Digital Twin Pipeline for Photorealistic Robot Simulation: Evaluating VLA Policy Deployment on a Bimanual Mobile Robot

Digital twins that are automatically constructed from robot sensor data offer a promising pathway for scalable Real2Sim and Sim2Real transfer. However, it remains an open question whether photorealistic reconstruction alone is sufficient to support reliable deployment of vision-language-action (VLA) policies. We present a generative-AI-assisted Real2Sim pipeline that generates simulation-ready digital twins from real-world RGB observations with minimal manual intervention. The pipeline uses prompted segmentation to isolate scene components and a generative 3D model to directly produce simulation assets, eliminating the need for traditional multi-view reconstruction or manual 3D modeling.\r\nTo evaluate simulation fidelity, we deploy and compare policies from two VLA models in both the real robot and the reconstructed\r\nsimulation under identical tasks and initial conditions. We compare joint-level action trajectories and analyze how divergence evolves over time in closed-loop execution. Although the reconstructed environments are visually accurate, we observe increasing trajectory divergence during closedloop operation. These results indicate that photorealistic reconstruction alone is insufficient to preserve closed-loop control behavior\r\nin VLA policies, particularly in contact-rich manipulation settings where small perceptual errors compound over time.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Approach for energy efficient building design during early phase of design process

Energy consumption in the building sector is about 40% of total energy consumed globally and is trending upwards, along with its contribution to greenhouse gas (GHG) emissions. Given the adverse impacts of GHG emissions, it is crucial to integrate energy efficiency into building designs. The most significant opportunities for enhancing energy performance are present during the initial phases of building design, when there is less impact of other design constraints. Various tools exist for simulating different design options and providing feedback in terms of energy consumption and comfort parameters. These simulation outputs must then be analyzed to derive design solutions. This paper presents an innovative approach that utilizes user input parameters, processes them through cloud computing, and outputs easily understandable strategies for energy-efficient building design. The methodology employs Asynchronous Distributed Task Queues (DTQ) - a more scalable and reliable alternative to conventional speedup techniques-for conducting parametric energy simulations in the cloud. The goal of this approach is to assist design teams in identifying, visualizing, and prioritizing energy-saving design strategies from a range of possible solutions for each project. Furthermore, a tool ‘eDOT’ has been developed utilizing the discussed methodology. Unlike existing tools, eDOT leverages artificial intelligence to dynamically generate and provide design strategies during the early phases of design process. By simplifying the simulation process, eDOT enables design teams to make informed, data-driven decisions without needing to interpret complex simulation outputs. A case study simulated for two locations is provided in this paper to demonstrate the effectiveness of eDOT, further underscoring its practical impact on energy-efficient building design.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2025

AI foundation models for experimental fusion tasks

Artificial Intelligence (AI) foundation models, while successful in various domains of language, speech, and vision, have not been adopted in production for fusion energy experiments. This brief paper presents how AI foundation models can be used for fusion energy diagnostics, enabling, for example, visual automated logbooks to provide greater insights into chains of plasma events in a discharge, in time for between-shot analysis.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2024

Unsupervised multimodal fusion of in-process sensor data for advanced manufacturing process monitoring

Effective monitoring of manufacturing processes is crucial for maintaining product quality and operational efficiency. Modern manufacturing environments often generate vast amounts of complementary multimodal data, including visual imagery from various perspectives and resolutions, hyperspectral data, and machine health monitoring information such as actuator positions, accelerometer readings, and temperature measurements. However, fusing and interpreting this complex, high-dimensional data presents significant challenges, particularly when labeled datasets are unavailable or impractical to obtain. This paper presents a novel approach to multimodal sensor data fusion in manufacturing processes, inspired by the Contrastive Language-Image Pre-training (CLIP) model. We leverage contrastive learning techniques to correlate different data modalities without the need for labeled data, overcoming limitations of traditional supervised machine learning methods in manufacturing contexts. Our proposed method demonstrates the ability to handle and learn encoders for five distinct modalities: visual imagery, audio signals, laser position (x and y coordinates), and laser power measurements. By compressing these high-dimensional datasets into low-dimensional representational spaces, our approach facilitates downstream tasks such as process control, anomaly detection, and quality assurance. The unsupervised nature of our method makes it broadly applicable across various manufacturing domains, where large volumes of unlabeled sensor data are common. We evaluate the effectiveness of our approach through a series of experiments, demonstrating its potential to enhance process monitoring capabilities in advanced manufacturing systems. This research contributes to the field of smart manufacturing by providing a flexible, scalable framework for multimodal data fusion that can adapt to diverse manufacturing environments and sensor configurations. The proposed method paves the way for more robust, data-driven decision-making in complex manufacturing processes.

Contrastive Learning↗

DOE OSTI2024

Advancing Building Energy Modeling with Large Language Models: Exploration and Case Studies

The rapid progression in artificial intelligence has facilitated the emergence of large language models like ChatGPT, offering potential applications extending into specialized engineering modeling, especially physics-based building energy modeling. This paper investigates the innovative integration of large language models with building energy modeling software, focusing specifically on the fusion of ChatGPT with EnergyPlus. A literature review is first conducted to reveal a growing trend of incorporating large language models in engineering modeling, albeit limited research on their application in building energy modeling. We underscore the potential of large language models in addressing building energy modeling challenges and outline potential applications including simulation input generation, simulation output analysis and visualization, conducting error analysis, co-simulation, simulation knowledge extraction and training, and simulation optimization. Three case studies reveal the transformative potential of large language models in automating and optimizing building energy modeling tasks, underscoring the pivotal role of artificial intelligence in advancing sustainable building practices and energy efficiency. The case studies demonstrate that selecting the right large language model techniques is essential to enhance performance and reduce engineering efforts. The findings advocate a multidisciplinary approach in future artificial intelligence research, with implications extending beyond building energy modeling to other specialized engineering modeling.

building energy modeling↗

DOE OSTI2025

Reimagining Disassembly Interfaces With Visualization: Combining Instruction Tracing and Control Flow With DisViz

In applications where efficiency is critical, developers may examine their compiled binaries, seeking to understand how the compiler transformed their source code and what performance implications that transformation may have. This analysis is challenging due to the vast number of disassembled binary instructions and the many-to-many mappings between them and the source code. These problems are exacerbated as source code size increases, giving the compiler more freedom to map and disperse binary instructions across the disassembly space. Interfaces for disassembly typically display instructions as an unstructured listing or sacrifice the order of execution. Here, we design a new visual interface for disassembly code that combines execution order with control flow structure, enabling analysts to both trace through code and identify familiar aspects of the computation. Central to our approach is a novel layout of instructions grouped into basic blocks that displays a looping structure in an intuitive way. We add to this disassembly representation a unique block-based mini-map that leverages our layout and shows context across thousands of disassembly instructions. Finally, we embed our disassembly visualization in a web-based tool, DisViz, which adds dynamic linking with source code across the entire application. DizViz was developed in collaboration with program analysis experts following design study methodology and was validated through evaluation sessions with ten participants from four institutions. Participants successfully completed the evaluation tasks, hypothesized about compiler optimizations, and noted the utility of our new disassembly view. Our evaluation suggests that our new integrated view helps application developers in understanding and navigating disassembly code.

Computer science↗

DOE OSTI2025

ARCH: Large-scale knowledge graph via aggregated narrative codified health records analysis

Objective: Electronic health record (EHR) systems contain a wealth of clinical data stored as both codified data and free-text narrative notes (NLP). The complexity of EHR presents challenges in feature representation, information extraction, and uncertainty quantification. Here, to address these challenges, we proposed an efficient Aggregated naRrative Codified Health (ARCH) records analysis to generate a large-scale knowledge graph (KG) for a comprehensive set of EHR codified and narrative features. Methods: Using data from 12.5 million Veterans Affairs patients, ARCH first derives embedding vectors and generates similarities along with associated p-values to measure the strength of relatedness between clinical features with statistical certainty quantification. Next, ARCH performs a sparse embedding regression to remove indirect linkage between features to build a sparse KG. Finally, ARCH was validated on various clinical tasks, including detecting known relationships between entity pairs, predicting drug side effects, disease phenotyping, as well as sub-typing Alzheimer’s disease patients. Results: ARCH produces high-quality clinical embeddings and KG for over 60,000 codified and narrative EHR concepts. The KG and embeddings are visualized in the R-shiny powered web-API.3 ARCH achieved high accuracy in detecting EHR concept relationships, with AUCs of 0.926 (codified) and 0.861 (NLP) for similar EHR concepts, and 0.810 (codified) and 0.843 (NLP) for related pairs. It detected drug side effects with a 0.723 AUC, which improved to 0.826 after fine-tuning. Using both codified and NLP features, the detection power increased significantly. Compared to other methods, ARCH has superior accuracy and enhances weakly supervised phenotyping algorithms’ performance. Notably, it successfully categorized Alzheimer’s patients into two subgroups with varying mortality rates. Conclusion: The proposed ARCH algorithm generates large-scale high-quality semantic representations and knowledge graph for both codified and NLP EHR features, useful for a wide range of predictive modeling tasks.

Electronic health records↗

DOE OSTI2025

Using Eye Tracking to Elucidate the Mechanisms Underlying Stimulation-Enhanced Visual Target Detection

Transcranial direct current stimulation (tDCS) is a noninvasive form of brain stimulation that involves passing a weak electrical current between electrodes on the scalp to modulate underlying neural tissue. TDCS has been shown to modulate cognition in a variety of domains, including memory, attention, and visual processing. Prior work from our laboratory has shown positive effects of tDCS on learning to detect target objects hidden in complex naturalistic visual scenes and learn rules for categorizing images, though the mechanism for these benefits remains unknown. One possibility is that tDCS optimizes visual search by modulating visual attention or via the reduction in search errors. One method of quantifying visual attention is to use eye tracking to record search patterns to determine if and how visual search is adjusted under verum stimulation conditions. Eye tracking data allows classification of errors into error types, including sampling errors (failing to look in the relevant region), recognition errors (looking at the critical portion of a scene, but failing to recognize it as such as evidenced by visual fixation), and decision-making errors (fixating on the relevant portion of a scene, but making the wrong determination). Our results indicate that the benefit tDCS confers on visual search for targets stems from the reduction in decision-making errors when targets are present (Cohen’s d = 0.86). Also reported is a replication of previous findings showing a tDCS-dependent improvement in learning this task, learning score (Cohen’s d = 0.88); d’ (Cohen’s d = 1.00). This provides support for moving tDCS into the application space by pairing it with analysts who are concerned with the type of search error that is corrected via stimulation.

attention↗

DOE OSTI2025

Ocelot: An Interactive, Efficient Distributed Compression-As-a-Service Platform With Optimized Data Compression Techniques

Large volumes of data generated by scientific simulations, genome sequencing, and other applications need to be moved among clusters for data collection/analysis. Data compression techniques have effectively reduced data storage and transfer costs. However, users' requirements on interactively controlling both data quality and compression ratios are non-trivial to fulfill. Here, we propose a novel Compression-as-a-Service (CaaS) platform called Ocelot with four important contributions: (1) It offers real-time visualization, interactive compression, and transfer of scientific datasets. (2) It incorporates new strategies for compressing diverse types of datasets more effectively than traditional methods. (3) It provides an effective method for estimating the compression ratio and execution time of compression tasks. (4) Experiments on multiple real-world datasets on geographically distributed computers show that Ocelot can significantly improve data transfer efficiency with a performance gain of more than 10x in computing clusters with relatively slow networks.

compression as a service (CaaS)↗

DOE OSTI2026

CMS Outer Tracker Module QA

The High Luminosity Large Hadron Collider is a planned upgrade to the LHC that will allow it to take up to 7 times as much data. Upgrades to all detector systems, including the Outer Tracker, are necessary to allow them to handle the increased radiation and pileup. Fermilab has been tasked with assembling modules for the Outer Tracker system. This presentation discusses the QA tests performed on these modules and the integration procedure required to ensure all modules work in sync. The QA tests include sensor IV curves, MaPSA testing, visual inspection, skeleton testing, and module testing. There are additional tests that can only be performed on integrated modules that are working in sync, and developing an understanding of the timing and latency of the modules is necessary to perform this synchronization.

Hobbeheydar, Samuel [Fermilab; Carnegie Mellon U. ↗

DOE OSTI2026

CMS Outer Tracker Module QA

Hobbeheydar, Samuel [Fermilab; Carnegie Mellon U. ↗

DOE OSTI2024

In-Line Optical Transmission Imaging of Decals for Quality Control - Task 3

Quality monitoring is a critical aspect for manufacturing systems. Ideally the monitoring would be done in-line, be non-contact, non-destructive, and fast. This would enable reduced scrap and higher throughput. This poster presents an optical transmission method for evaluating and mapping coatings. With the method shown in the poster we can visualize optical variations on the macro and micro scales. This allows us to see the overall trend in loading in both the cross web and down web directions. Furthermore, we can visualize defects such dewetting spots, streaks, clumps, and pinholes where there is a lack of coating. The optical transmission signal has been found to be proportional to the IrOx loading signal using XRF measurements. Therefore, an optical transmission setup can be installed in-line and allow for a fast, non-contact method for mapping loading variations and defects.

coating uniformity↗

DOE OSTI2026

String-Breaking Dynamics in Quantum Adiabatic and Diabatic Processes

Confinement prohibits isolation of color charges, e.g., quarks, in nature via a process called string breaking : the separation of two charges results in an increase in the energy of a color flux, visualized as a string, connecting those charges. Eventually, creating additional charges is energetically favored, hence breaking the string. Such a phenomenon can be probed in simpler models, including quantum spin chains, enabling enhanced understanding of string-breaking dynamics. A challenging task is to understand how string breaking occurs as time elapses, in an out-of-equilibrium setting. This work establishes the phenomenology of dynamical string breaking induced by a gradual increase of string tension over time. It, thus, goes beyond instantaneous quench processes and enables tracking the real-time evolution of strings in a more controlled setting. We focus on domain-wall confinement in a family of quantum Ising chains. Our results indicate that, for sufficiently short strings and slow evolution, string breaking can be described by the transition dynamics of a two-state quantum system akin to a Landau-Zener process. For longer strings, a more intricate spatiotemporal pattern emerges: the string breaks by forming a superposition of bubbles (domains of flipped spins of varying sizes), which involve highly excited states. We finally demonstrate that string breaking driven only by quantum fluctuations can be realized in the presence of sufficiently long-ranged interactions. This work holds immediate relevance for studying string breaking in quantum-simulation experiments.

Ising model↗

DOE OSTI2025

Extremely Scalable Distributed Computation of Contour Trees via Pre-Simplification

Contour trees offer an abstract representation of the level set topology in scalar fields and are widely used in topological data analysis and visualization. However, applying contour trees to large-scale scientific datasets remains challenging due to scalability limitations. Recent developments in distributed hierarchical contour trees have addressed these challenges by enabling scalable computation across distributed systems. Building on these structures, advanced analytical tasks—such as volumetric branch decomposition and contour extraction—have been introduced to facilitate large-scale scientific analysis. Despite these advancements, such analytical tasks substantially increase memory usage, which hampers scalability. In this paper, we propose a pre-simplification strategy to significantly reduce the memory overhead associated with analytical tasks on distributed hierarchical contour trees. We demonstrate enhanced scalability through strong scaling experiments, constructing the largest known contour tree—comprising over half a trillion nodes with complex topology—in under 15 minutes on a dataset containing 550 billion elements.

Li, Mingzhe [University of Utah]↗

DOE OSTI2026

Vision and Development of a Design, Implementation, and Verification Automation (DIVA) Software Platform for DNA Construction

Abstract DNA construction, while a prerequisite to many biological endeavors, is often a time-consuming distraction from an individual’s primary research objectives. We envisioned that with the right software infrastructure and cultural mindset, a single person could execute in parallel the batched DNA construction tasks of an entire research institute, at scales realizing efficiency gains through process and laboratory automation. In pursuit of this vision, we developed the Design, Implementation, and Verification Automation (DIVA) software platform. DIVA’s web interface enables researchers to design DNA constructs (using visual biological computer-aided design tools and biological parts repositories), submit designs for construction to dedicated staff, and track DNA construction as it progresses. DIVA supports the dedicated staff through the DNA construction process and records both successful and unsuccessful attempts toward improving the overall process. The platform is publicly available at public-diva.jbei.org and its open-source code through github.com/JBEI/DIVA.

Plahar, Hector [DOE Agile BioFoundry , , ,; DOE Jo↗

DOE OSTI2026

WM26 Paper Multi-Robot Collaboration for Hazardous Environments

Hazardous nuclear and industrial facilities are rarely designed for robots. Work in these domains demand precise manipulation and robust mobility in cluttered, constrained spaces where off-the-shelf platforms struggle and “one-size-fits-all” machines become costly and complex. Idaho National Laboratory (INL) is developing an autonomous, multi-robot inspection system that coordinates task-specific platforms rather than relying on a single omni-tool robot. An electric truck serves as a power and compute hub for a custom manipulator co-developed with Florida International University (FIU), a commercial mini crawler, a pan–tilt–zoom camera, and a Nexxis Argus LiDAR mapping system. Working in concert, these robots generate spatial, radiation, and temperature maps of the pit environments at the Hanford Waste Tank Farms. These systems will capture visual records and environmental telemetry to allow for analysis post inspection. The system architecture uses Robot Operating System 2 (ROS 2) for publish/subscribe integration, NVIDIA Isaac Sim and Unity for simulation and visualization, and algorithms such as NVBlox to fuse data into unified 3D overlays. This robot-agnostic approach reduces operator burden by enabling autonomy across heterogeneous platforms and lets each robot be used where it is strongest. Having autonomous functions means operators don’t have to fully control multiple different components. The ease of use could allow for more widespread adoption of advanced robotics at waste management sites that see continued use. By coordinating simpler, purpose-built mechanisms, the approach lowers design and manufacturing complexity, reduces capital risk in contaminated settings, and improves controllability for complex inspection and manipulation tasks. We present the architecture, early results, and lessons learned from building and deploying this coordinated multi-robot system, with the goal of accelerating safe, cost-effective adoption of advanced robotics at waste-management sites.

42 - ENGINEERING↗

DOE OSTI2024

Accelerating Multivariate Functional Approximation Computation with Domain Decomposition Techniques⋆

Modeling large datasets through Multivariate Functional Approximations (MFA) provide an elegant way to handle many visualization and scientific analysis workflows. The process necessitates scalable data partitioning methods to compute MFA representations efficiently without compromising the accuracy or continuity of the reconstructed solution. We propose a domain -decomposed method for computing the MFA with B -spline bases, which reduces the total work per task and uses a restricted Additive Schwarz (RAS) method to converge the control point data degrees -of -freedom along subdomain boundaries. We provide an in-depth analysis of the parallel approach with domain decomposition solvers, aiming to minimize local subdomain error residuals and recover high -order continuity at subdomain interfaces with appropriate choices of knot overlaps. The communication cost, determined by the overlap regions in the RAS implementation, is optimized to recover the numerical error profile of the single subdomain case. Our proposed method stands in contrast to previous methods, which typically only recover either C 0 or at best C 1 continuity for arbitrary B -spline degree expansions, or those that require post -processing to blend discontinuities in the reconstructed data. We demonstrate the effectiveness of our approach using analytical and real -world datasets in 1D, 2D, and 3D through both strong and weak scaling studies. The performance results indicate that the overall cost of computing the approximation is directly proportional to the underlying nearest -neighbor communication implementation, and is only weakly dependent on the overlap region size that determines the size of the messages. This finding underscores the efficiency and scalability of our proposed method, making it a promising solution for handling large datasets in scientific workflows.

additive Schwarz solvers↗

DOE OSTI2024

Streamlining latent spaces in machine learning using moment pooling

Many machine learning applications involve learning a latent representation of data, which is often high-dimensional and difficult to directly interpret. In this work, we propose “moment pooling,” a natural extension of deep sets networks which drastically decreases the latent space dimensionality of these networks while maintaining or even improving performance. Moment pooling generalizes the summation in deep sets to arbitrary multivariate moments, which enables the model to achieve a much higher effective latent dimensionality for a fixed learned latent space dimension. We demonstrate moment pooling on the collider physics task of quark/gluon jet classification by extending energy flow networks (EFNs) to moment EFNs. We find that moment EFNs with latent dimensions as small as 1 perform similarly to ordinary EFNs with higher latent dimension. This small latent dimension allows for the internal representation to be directly visualized and interpreted, which in turn enables the learned internal jet representation to be extracted in closed form. Published by the American Physical Society 2024

Gambhir, Rikab (ORCID:0000000251080448)↗