Search NASA⌕ Search

SEARCH · Search NASA

Results for “Human Computer Interaction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Application of Artificial Intelligence/Machine Learning to Operations Research

This report examines the transformative impact of Artificial Intelligence (AI) and Machine Learning (ML) on operations research, private industry, and government sectors, highlighting their applications in automating processes, enhancing decision-making, and optimizing complex systems. AI/ML technologies have revolutionized industries through predictive maintenance, supply chain optimization, and autonomous systems, while also advancing public safety and defense operations. However, challenges such as data integrity, model transparency, and the need for human oversight persist, particularly in high-consequence environments. The report emphasizes the critical role of explainable AI (XAI) and human-computer interaction models like Human-in-the-Loop (HITL) and Human-on-the-Loop (HOTL) in fostering trust and accountability. Balancing automation with ethical responsibility and transparency is essential for the continued successful integration of AI/ML into operational and strategic decision-making frameworks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Developing Digital Twin Visualizations: A Methodology and Case Study on Chemical Separation Processing

As advances in digital engineering continue to push the technological boundaries, digital twin (DT) visualizations for diagnostics and safeguards advancement become much more feasible and practical. DTs generate large and complex data streams that require effective user interfaces to provide monitoring and diagnostic capabilities. Unfortunately, while these frameworks exist, there is not much research on the systematic documentation of human–computer interaction (HCI) for DT visualization. This work presents a dual-mode visualization methodology (two dimensional [2D] graphical user interface dashboard and 3D mixed reality) designed to support diagnostic tasks in DT systems and building on a validated framework and applying established HCI principles. The methodology is demonstrated through a case study of aqueous processing at Idaho National Laboratory, using experimental data from the chemical solvent extraction runs. Our interfaces display real-time alerts and monitoring to inform users of safeguards anomalies. The interfaces use immersive 3D mixed-reality visualization for further system and experiment investigation. This work demonstrates how the systematic application of HCI principles can inform DT visualization design for diagnostic and safeguards applications. While formal user evaluation studies remain as future work, this paper documents the systematic design methodology and demonstrates a proof-of-concept implementation.

3D visualization↗
DOE OSTI2025

ICAT: The Interactive Corpus Analysis Tool

The Interactive Corpus Analysis Tool (ICAT) is a Python library for creating dashboards to explore textual datasets and build simple binary classification models to help filter through them and focus on entries of interest. This tool uses a form of interactive machine learning (IML), a paradigm of “machine teaching” (Simard et al., 2017) that sits at the intersection of the fields of human computer interaction (HCI), visual analytics, and machine learning. The intent of ICAT is to allow subject matter experts (SME) with limited to no experience in machine learning to benefit from an iterative human-in-the-loop (HITL) approach to building their own model without needing to understand the details of the underlying algorithm. This interactivity is achieved by allowing the user to create features, label data points, and visually manipulate a representation of the features to manually cluster and investigate data, while a model is trained on the fly based on these actions. ICAT is built on top of the Panel (Holoviz, 2018) library, using a combination of Vega, a custom IPyWidget using D3, and ipyvuetify, and is intended to be used inside of a Jupyter environment.

Martindale, Nathan [Oak Ridge National Laboratory ↗
DOE OSTI2025

VISION: a modular AI assistant for natural human-instrument interaction at scientific user facilities

Scientific user facilities, such as synchrotron beamlines, are equipped with a wide array of hardware and software tools that require a codebase for human-computer-interaction. This often necessitates developers to be involved to establish connection between users/researchers and the complex instrumentation. The advent of generative AI presents an opportunity to bridge this knowledge gap, enabling seamless communication and efficient experimental workflows. Here we present a modular architecture for the Virtual Scientific Companion by assembling multiple AI-enabled cognitive blocks that each scaffolds large language models (LLMs) for a specialized task. With VISION, we performed LLM-based operation on the beamline workstation with low latency and demonstrated the first voice-controlled experiment at an x-ray scattering beamline. The modular and scalable architecture allows for easy adaptation to new instruments and capabilities. Development on natural language-based scientific experimentation is a building block for an impending future where a science exocortex—a synthetic extension to the cognition of scientists—may radically transform scientific practice and discovery.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Large language model evaluation for high–performance computing software development

We apply AI-assisted large language model (LLM) capabilities of GPT-3 targeting high-performance computing (HPC) kernels for (i) code generation, and (ii) auto-parallelization of serial code in C ++, Fortran, Python and Julia. Our scope includes the following fundamental numerical kernels: AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG, and language/programming models: (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numpy, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). Kernel implementations are generated using GitHub Copilot capabilities powered by the GPT-based OpenAI Codex available in Visual Studio Code given simple + + prompt variants. To quantify and compare the generated results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. For auto-parallelization, we use ChatGPT interactively giving simple prompts as in a dialogue with another human including simple “prompt engineering” follow ups. Results suggest that correct outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general-purpose Python can benefit from adding language keywords, while Julia prompts perform acceptably well for its Threads and CUDA.jl programming models. Finally, we expect to provide an initial quantifiable point of reference for code generation in each programming model using a state-of-the-art LLM. Overall, understanding the convergence of LLMs, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human-computer interactions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Radsource Mr: Mixed Reality Planning Tool For Radioactive Recovery

The RadSource MR system leverages Meta Quest 3's advanced mixed reality capabilities to create a comprehensive spatial planning platform for end-of-life sealed radioactive source recovery operations. The application utilizes the Quest 3's high-resolution passthrough cameras and spatial mapping algorithms to generate accurate 3D environmental models. Core technical components include: (1) Real-time spatial measurement algorithms calculating distances, angles, slopes, and surface areas with sub-centimeter accuracy; (2) Virtual object placement system allowing users to position digital representations of recovery equipment (trailers, containment vessels, protective barriers) within the real environment; (3) Voice recording and annotation system for hands-free documentation in protective equipment; (4) 3D mesh capture and storage capabilities for post-operation analysis and regulatory documentation. (5) Procedure documentation is available for viewing in Mixed Reality, providing an innovative and convenient way to access the information during pre-visit and inspection activities. (6) Support for screen capture for the view for real world and virtual objects together to use it later for planning. The system integrates computer vision techniques for environmental understanding, spatial mathematics for precise measurements, and human-computer interaction principles optimized for hazardous environment operations. Data persistence allows teams to save and share planning sessions across multiple stakeholders while maintaining operational security requirements.

Khadka, Rajiv [Idaho National Laboratory (INL), Id↗
DOE OSTI2025

Exocortex Network for AI-Augmented Human-Led Scientific Expedition

AI advances in science can be viewed along two main directions with a fluid boundary: enhancing efficiency through automation and smart tools to accelerate tasks that humans can already perform; and enabling exploration into uncharted territories and potentially toward AGI. These advances manifest in the AI cognitive core through the development and explainability of foundation models; in the physical embodiment of instruments and facilities; and in the integrated agency of AI workflows exemplified by the science exocortex. To address the role of humans in this evolving landscape, in this Perspective, we suggest a third direction: the development of personalized agents that form human-centered networks, supporting both efficiency and exploration while ensuring that AI remains aligned with human vision.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Architecture for Web-Based Visualization of Large-Scale Energy Domains: Preprint

With the growing penetration of inverter-based distributed energy resources and increased loads through electrification, power systems analyses are becoming more important and more complex. Moreover, these analyses increasingly involve the combination of interconnected energy domains with data that are spatially and temporally increasing in scale by orders of magnitude, surpassing the capabilities of many existing analysis and decision-support systems. We present the architectural design, development, and application of a high-resolution web-based visualization environment capable of cross-domain analysis of tens of millions of energy assets, focusing on scalability and performance. Our system supports the exploration, navigation, and analysis of large data from diverse domains such as electrical transmission and distribution systems, mobility and electric vehicle charging networks, communications networks, cyber assets, and other supporting infrastructure. We evaluate this system across multiple use cases, describing the capabilities and limitations of a web-based approach for high-resolution energy system visualizations.

grid modernization↗
DOE OSTI2024

An open-source data storage and visualization platform for collaborative qubit control

Developing collaborative research platforms for quantum bit control is crucial for driving innovation in the field, as they enable the exchange of ideas, data, and implementation to achieve more impactful outcomes. Furthermore, considering the high costs associated with quantum experimental setups, collaborative environments are vital for maximizing resource utilization efficiently. However, the lack of dedicated data management platforms presents a significant obstacle to progress, highlighting the necessity for essential assistive tools tailored for this purpose. Current qubit control systems are unable to handle complicated management of extensive calibration data and do not support effectively visualizing intricate quantum experiment outcomes. In this paper, we introduce Qubit Control Storage and Visualization ( QubiCSV ), a platform specifically designed to meet the demands of quantum computing research, focusing on the storage and analysis of calibration and characterization data in qubit control systems. As an open-source tool, QubiCSV facilitates efficient data management of quantum computing, providing data versioning capabilities for data storage and allowing researchers and programmers to interact with qubits in real time. The insightful visualization are developed to interpret complex quantum experiments and optimize qubit performance. QubiCSV not only streamlines the handling of qubit control system data but also improves the user experience with intuitive visualization features, making it a valuable asset for researchers in the quantum computing domain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Interactive Gas Chemistry for Enhanced Science Capabilities of the Energy Exascale Earth System Model Version 3

Atmospheric chemistry plays a crucial role in Earth system models (ESMs), controlling atmospheric composition and radiative balance; it is highly interactive with the physical climate, biogeochemical cycles, and human systems. However, it often imposes computational challenges in an ESM. Here we develop a full troposphere‐stratosphere interactive chemistry module for the US Department of Energy's Energy Exascale Earth System Model (E3SM). We intentionally build a streamlined module based on E3SM version 2 that interacts with other components and maintains all of major chemical and chemistry‐climate feedbacks. The module incorporates a new, highly efficient tracer advection scheme; linearization of stratospheric chemistry; and abridged tropospheric chemical mechanism with 28 reactive tracers. This new model, E3SM‐chem, can readily perform century‐long climate simulations of ozone, methane, and nitrous oxide based on emission scenarios as well as provide hourly budgets for the gas‐phase radicals that drive aerosol chemistry. We evaluate E3SM‐chem with an atmosphere‐only simulation as in the recent climate model intercomparison project (CMIP6) finding results similar to the other CMIP6 models. For the present‐day, E3SM‐chem matches the standard measurement metrics for stratospheric and tropospheric ozone, surface air quality, other key reactive gases like carbon monoxide, and the methane lifetime. Overall, E3SM‐chem maintains the climate fidelity of the baseline model while adding at most 20% to the computational cost of the atmosphere model. Hence, interactive chemistry can be a default configuration for long climate simulations at resolutions of 1° or finer, which is crucial for producing self‐consistent chemistry‐climate feedbacks that alter the climate system.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

DiffESM: Conditional Emulation of Temperature and Precipitation in Earth System Models With 3D Diffusion Models

Earth system models (ESMs) are essential for understanding the interaction between human activities and the Earth's climate. However, the computational demands of ESMs often limit the number of simulations that can be run, hindering the robust analysis of risks associated with extreme weather events. While low-cost climate emulators have emerged as an alternative to emulate ESMs and enable rapid analysis of future climate, many of these emulators only provide output on at most a monthly frequency. This temporal resolution is insufficient for analyzing events that require daily characterization, such as heat waves or heavy precipitation. We propose using diffusion models, a class of generative deep learning models, to effectively downscale ESM output from a monthly to a daily frequency. Trained on a handful of ESM realizations, reflecting a wide range of radiative forcings, our DiffESM model takes monthly mean precipitation or temperature as input, and is capable of producing daily values with statistical characteristics close to ESM output. Combined with a low-cost emulator providing monthly means, this approach requires only a small fraction of the computational resources needed to run a large ensemble. We evaluate model behavior using a number of extreme metrics, showing that DiffESM closely matches the spatio-temporal behavior of the ESM output it emulates in terms of the frequency and spatial characteristics of phenomena such as heat waves, dry spells, or rainfall intensity.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

A multi-scale cognitive interaction model of instrument operations at the Linac Coherent Light Source

The Linac Coherent Light Source (LCLS) is the world’s first x-ray free electron laser. It is a scientific user facility operated by the SLAC National Accelerator Laboratory, at Stanford, for the U.S. Department of Energy. As beam time at LCLS is extremely valuable and limited, experimental efficiency—getting the most high quality data in the least time—is critical. Our overall project employs cognitive engineering methodologies with the goal of improving experimental efficiency and increasing scientific productivity at LCLS by refining experimental interfaces and workflows, simplifying tasks, reducing errors, and improving operator safety and stress. Here, in this study, we describe a multi-agent, multi-scale computational cognitive interaction model of instrument operations at LCLS. Our model simulates the aspects of human cognition at multiple cognitive and temporal scales, ranging from seconds to hours, and among agents playing multiple roles, including instrument operator, real time data analyst, and experiment manager. The model can roughly predict impacts stemming from proposed changes to operational interfaces and workflows. Example results demonstrate the model’s potential in guiding modifications to improve operational efficiency. We discuss the implications of our effort for cognitive engineering in complex experimental settings and outline future directions for research. The model is open source, and the videos of the supplementary material provide extensive detail.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Beyond microbial abundance: metadata integration enhances disease prediction in human microbiome studies

Multiple studies have highlighted the interaction of the human microbiome with physiological systems such as the gut, immune, liver, and skin, via key axes. Advances in sequencing technologies and high-performance computing have enabled the analysis of large-scale metagenomic data, facilitating the use of machine learning to predict disease likelihood from microbiome profiles. However, challenges such as compositionality, high dimensionality, sparsity, and limited sample sizes have hindered the development of actionable models. One strategy to improve these models is by incorporating key metadata from both the human host and sample collection/processing protocols. This remains challenging due to sparsity and inconsistency in metadata annotation and availability. In this paper, we introduce a machine learning-based pipeline for predicting human disease states by integrating host and protocol metadata with microbiome abundance profiles from 68 different studies, processed through a consistent pipeline. Our findings indicate that metadata can enhance machine learning predictions, particularly at higher taxonomic ranks like Kingdom and Phylum, though this effect diminishes at lower ranks. Our study leverages a large collection of microbiome datasets comprising 11,208 samples, therefore enhancing the robustness and statistical confidence of our findings. This work is a critical step toward utilizing microbiome and metadata for predicting diseases such as gastrointestinal infections, diabetes, cancer, and neurological disorders.

Mathematics and Computing↗
DOE OSTI2025

Foundation Models for Zero-Shot Segmentation of Scientific Images without AI-Ready Data

Zero-shot and prompt-based models have excelled at visual reasoning tasks by leveraging large-scale natural image corpora, but they often fail on sparse and domain-specific scientific image data. We introduce Zenesis, a no-code interactive computer vision platform designed to reduce data readiness bottlenecks in scientific imaging workflows. Zenesis integrates lightweight multimodal adaptation for zero-shot inference on raw scientific data, human-in-the-loop refinement, and heuristic-based temporal enhancement. We validate our approach on Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) datasets of catalyst-loaded membranes. Zenesis outperforms baselines, achieving an average accuracy of 0.947, Intersection over Union (IoU) of 0.858, and Dice score of 0.923 on amorphous catalyst samples; and 0.987 accuracy, 0.857 IoU, and 0.923 Dice on crystalline samples. These results represent a significant performance gain over conventional methods such as Otsu thresholding and standalone models like the Segment Anything Model (SAM). Zenesis enables effective image segmentation in domains where annotated datasets are limited, offering a scalable solution for scientific discovery.

Mukherjee, Shubhabrata↗
DOE OSTI2025

G2PDeep-v2: A Web-Based Deep-Learning Framework for Phenotype Prediction and Biomarker Discovery for All Organisms Using Multi-Omics Data

Multi-omics data offers rich insights into complex traits across organisms, yet integrating and analyzing these datasets for phenotype prediction and marker discovery remains challenging. Researchers need accessible tools that combine deep learning, hyperparameter optimization, visualization, and downstream analysis in a unified web platform. To address this, we developed G2PDeep-v2, a web-based platform powered by deep learning for phenotype prediction and marker discovery from multi-omics data across a wide range of organisms, including humans and plants. The server provides multiple services for researchers to create deep-learning models through an interactive interface and train these models using an automated hyperparameter tuning algorithm on high-performance computing resources. Users can visualize the results of phenotype and markers predictions and perform Gene Set Enrichment Analysis for the significant markers to provide insights into the molecular mechanisms underlying complex diseases, conditions and other biological phenotypes being studied.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Toward Accelerating Discovery via Physics-Driven and Interactive Multifidelity Bayesian Optimization

Both computational and experimental material discovery bring forth the challenge of exploring multidimensional and often nondifferentiable parameter spaces, such as phase diagrams of Hamiltonians with multiple interactions, composition spaces of combinatorial libraries, processing spaces, and molecular embedding spaces. Often these systems are expensive or time consuming to evaluate a single instance, and hence classical approaches based on exhaustive grid or random search are too data intensive. This resulted in strong interest toward active learning methods such as Bayesian optimization (BO) where the adaptive exploration occurs based on human learning (discovery) objective. However, classical BO is based on a predefined optimization target, and policies balancing exploration and exploitation are purely data driven. In practical settings, the domain expert can pose prior knowledge of the system in the form of partially known physics laws and exploration policies often vary during the experiment. Here, we propose an interactive workflow building on multifidelity BO (MFBO), starting with classical (data-driven) MFBO, then expand to a proposed structured (physics-driven) structured MFBO (sMFBO), and finally extend it to allow human-in-the-loop interactive interactive MFBO (iMFBO) workflows for adaptive and domain expert aligned exploration. These approaches are demonstrated over highly nonsmooth multifidelity simulation data generated from an Ising model, considering spin–spin interaction as parameter space, lattice sizes as fidelity spaces, and the objective as maximizing heat capacity. Detailed analysis and comparison show the impact of physics knowledge injection and real-time human decisions for improved exploration with increased alignment to ground truth. Here, the associated notebooks allow to reproduce the reported analyses and apply them to other systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

High-throughput methods leveraging robotics and computer vision for the development of therapeutic phage cocktails

We present the high-throughput automated screening techniques that are being used to develop bacteriophage-based therapeutic products currently under investigation in human clinical trials to combat urinary tract infections. By integrating modern liquid handling robotics, standardized phenotypic assays, and computer vision-based enumeration, we established a platform capable of reproducibly screening large collections of phages against clinically derived bacterial strain panels. This approach enabled systematic assessment of phage-bacteria interactions at scale, facilitating the identification and optimization of phage cocktails with broad in vitro activity. Although bacteriophage therapy has long been investigated as a strategy for treating bacterial infections, few frameworks exist for developing phage combinations in a reproducible and scalable manner. The methods outlined here address this gap and aim to support the broader development of therapeutic assets available to combat antibiotic resistance.

Penke, Taylor J. R. [Locus Biosciences, Morrisvill↗
DOE OSTI2024

Uncertainty-informed selection of CMIP6 Earth System Model subsets for use in multisectoral and impact models

Earth system models (ESMs) and general circulation models (GCMs) are heavily used to provide inputs to sectoral impact and multisector dynamic models, which include representations of energy, water, land, economics, and their interactions. Therefore, representing the full range of model uncertainty, scenario uncertainty, and interannual variability that ensembles of these models capture is critical to the exploration of the future co-evolution of the integrated human–Earth system. The pre-eminent source of these ensembles has been the Coupled Model Intercomparison Project (CMIP). With more modeling centers participating in each new CMIP phase, the size of the model archive is rapidly increasing, which can be intractable for impact modelers to effectively utilize due to computational constraints and the challenges of analyzing large datasets. In this work, we present a method to select a subset of the latest phase, CMIP6, featuring models for use as inputs to a sectoral impact or multisector dynamics models, while prioritizing preservation of the range of model uncertainty, scenario uncertainty, and interannual variability in the full CMIP6 ensemble results. This method is intended to help impact modelers select climate information from the CMIP archive efficiently for use in downstream models that require global coverage of climate information. This is particularly critical for large-ensemble experiments of multisector dynamic models that may be varying additional features beyond climate inputs in a factorial design, thus putting constraints on the number of climate simulations that can be used. We focus on temperature and precipitation outputs of CMIP6 models, as these are two of the most used variables among impact models, and many other key input variables for impacts are at least correlated with one or both of temperature and precipitation (e.g., relative humidity). Besides preserving the multi-model ensemble variance characteristics, we prioritize selecting CMIP6 models in the subset that preserve the very likely distribution of equilibrium climate sensitivity values as assessed by the latest Intergovernmental Panel on Climate Change (IPCC) report. This approach could be applied to other output variables of climate models and, possibly when combined with emulators, offers a flexible framework for designing more efficient experiments on human-relevant climate impacts. It can also provide greater insight into the properties of existing CMIP6 models.

Snyder, Abigail C.↗