Search NASA⌕ Search

SEARCH · Search NASA

Results for “Computer vision AI”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Vision Foundation Models in Remote Sensing: A survey

Artificial intelligence (AI) technologies have profoundly transformed the field of remote sensing (RS), revolutionizing data collection, processing, and analysis. Traditionally reliant on manual interpretation and task-specific models, RS research has been significantly enhanced by the advent of foundation models (FMs)—large-scale pretrained AI models capable of performing a wide array of tasks with unprecedented accuracy and efficiency. This article provides a comprehensive survey of FMs in the RS domain. We categorize these models based on their architectures, pretraining datasets, and methodologies. Through detailed performance comparisons, we highlight emerging trends and the significant advancements achieved by those FMs. Additionally, we discuss technical challenges, practical implications, and future research directions, addressing the need for high-quality data, computational resources, and improved model generalization. Our research also finds that pretraining methods, particularly self-supervised learning (SSL) techniques like contrastive learning (CL) and masked autoencoders (MAEs), remarkably enhance the performance and robustness of FMs. This survey aims to serve as a resource for researchers and practitioners by providing a panorama of advances and promising pathways for the continued development and application of FMs in RS.

data models↗
DOE OSTI2024

Workflow Provenance in the Computing Continuum for Responsible, Trustworthy, and Energy-Efficient AI

As Artificial Intelligence (AI) becomes more pervasive in our society, it is crucial to develop, deploy, and assess Responsible and Trustworthy AI (RTAI) models, i.e., those that consider not only accuracy but also other aspects, such as explainability, fairness, and energy efficiency. Workflow provenance data have historically enabled critical capabilities towards RTAI. Provenance data derivation paths contribute to responsible workflows through transparency in tracking artifacts and resource consumption. Provenance data are well-known for their trustworthiness helping explainability, reproducibility, and accountability. However, there are complex challenges to achieve RTAI, which are further complicated by the heterogeneous infrastructure in the computing continuum (Edge-Cloud-HPC) used to develop and deploy models. As a result, a significant research and development gap remains between workflow provenance data management and RTAI. In this paper, we present a vision of the pivotal role of workflow provenance in supporting RTAI and discuss related challenges. We present a schematic view between RTAI and provenance, and highlight open research directions.

Santos Souza, Renan↗
DOE OSTI2024

The VIBES Are Shifting: Assessing Emergent Capabilities in Multi-Modal Models

Researchers assessing open-source domains such as the internet, and particularly those studying information conflict, often have no single prescribed workflow. In the course of their research, they may need to perform a diverse array of tasks far beyond simply identifying an ever-changing set of objects. These analytical tasks can include ascertaining the provenance of images, understand an image in the context of accompanying text-based data, or identifying indicators of digital image manipulation. They must further be able to do this at the scale of tens of thousands of images or more. Traditional machine vision models have typically lacked the flexibility and breadth of performance sufficient for these needs. The research team from Pacific Northwest National Laboratory assessed the performance of a single baseline CLIP ViT-L model against a series of analytical tasks relevant for the study of online information conflict.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Micrometer: Micromechanics transformer for predicting full field mechanical responses of heterogeneous materials

Predicting mechanical responses of heterogeneous materials across scales remains a significant challenge. Traditional computational methods often struggle with complex and multiscale nature of these materials, limiting their effectiveness in real-world applications. Here, in this paper, we introduce Micrometer, a vision transformer based deep learning model designed to predict full field mechanical responses of heterogeneous materials, bridging the gap between computer vision and solid mechanics problems. We show that Micrometer, trained on a large-scale high-resolution dataset of 2D fiber-reinforced composites, can achieve state-of-the-art performance in predicting microscale strain fields across a wide range of material properties and loading conditions. Our model demonstrates accuracy and computational efficiency in applications such as computational homogenization and multiscale modeling, reducing computational time by up to two orders of magnitude compared to conventional numerical solvers while maintaining less than 1 % errors in predicting macroscale stress fields. Furthermore, we showcase Micrometer’s adaptability through transfer learning experiments on new materials with limited data, highlighting its potential to tackle diverse scenarios in computational solid mechanics. These results represent a significant step towards AI-driven innovation in materials science, addressing the limitations of traditional numerical methods and paving the way for more efficient simulations of heterogeneous materials across various industrial applications.

Composite materials↗
DOE OSTI2024

Machine learning materials properties with accurate predictions, uncertainty estimates, domain guidance, and persistent online accessibility

One compelling vision of the future of materials discovery and design involves the use of machine learning (ML) models to predict materials properties and then rapidly find materials tailored for specific applications. However, realizing this vision requires both providing detailed uncertainty quantification (model prediction errors and domain of applicability) and making models readily usable. At present, it is common practice in the community to assess ML model performance only in terms of prediction accuracy (e.g. mean absolute error), while neglecting detailed uncertainty quantification and robust model accessibility and usability. Here, we demonstrate a practical method for realizing both uncertainty and accessibility features with a large set of models. We develop random forest ML models for 33 materials properties spanning an array of data sources (computational and experimental) and property types (electrical, mechanical, thermodynamic, etc). All models have calibrated ensemble error bars to quantify prediction uncertainty and domain of applicability guidance enabled by kernel-density-estimate-based feature distance measures. All data and models are publicly hosted on the Garden-AI infrastructure, which provides an easy-to-use, persistent interface for model dissemination that permits models to be invoked with only a few lines of Python code. We demonstrate the power of this approach by using our models to conduct a fully ML-based materials discovery exercise to search for new stable, highly active perovskite oxide catalyst materials.

domain of applicability↗
DOE OSTI2025

RingX: Scalable Parallel Attention for Long-Context Learning on HPC

The attention mechanism has become foundational for remarkable AI breakthroughs since the introduction of the Transformer, driving the demand for increasingly longer context to power frontier models such as large-scale reasoning language models and high-resolution image/video generators. However, its quadratic computational and memory complexities present substantial challenges. Current state-of-the-art parallel attention methods, such as ring attention, are widely adopted for long-context training but utilize a point-to-point communication strategy that fails to fully exploit the capabilities of modern HPC network architectures. In this work, we propose ringX, a scalable family of parallel attention methods optimized explicitly for HPC systems. By enhancing workload partitioning, refining communication patterns, and improving load balancing, ringX achieves up to 3.4 × speedup compared to conventional ring attention on the Frontier supercomputer. Optimized for both bi-directional and causal attention mechanisms, ringX demonstrates its effectiveness through training benchmarks of a Vision Transformer (ViT) on a climate dataset and a Generative Pre-Trained Transformer (GPT) model, Llama3 8B. Our method attains an end-to-end training speedup of approximately 1.5 × in both scenarios. To our knowledge, the achieved 38% model FLOPs utilization (MFU) for training Llama3 8B with a 1M-token sequence length on 4,096 GPUs represents one of the highest training efficiencies reported for long-context learning on HPC systems. Our code implementation is available at https://github.com/jqyin/ringX-attention.

Yin, Junqi [ORNL] (ORCID:0000000338435520)↗
DOE OSTI2024

A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamics

AI-based foundation models like FourCastNet, GraphCast are revolutionizing weather and climate predictions but are not yet ready for operational use. Their limitation lies in the absence of a data assimilation system to incorporate real-time Earth system observations, crucial for accurately forecasting events like tropical cyclones. To overcome these obstacles, we introduce a generic real-time data assimilation framework and demonstrate its end-to-end performance on the Frontier supercomputer. This framework comprises two primary modules: an ensemble score filter (EnSF), which significantly outperforms the state-of-the-art data assimilation method, and a vision transformer-based surrogate capable of real-time adaptation through the integration of observational data. We demonstrate both the strong and weak scaling of our framework up to 1024 GPUs on the Exascale supercomputer, Frontier. Our results not only illustrate the framework's exceptional scalability on high-performance computing systems, but also demonstrate the importance of supercomputers in real-time data assimilation for weather and climate predictions.

Lu, Dan↗
DOE OSTI2023

FAIR for AI: An interdisciplinary and international community building perspective

A foundational set of findable, accessible, interoperable, and reusable (FAIR) principles were proposed in 2016 as prerequisites for proper data management and stewardship, with the goal of enabling the reusability of scholarly data. The principles were also meant to apply to other digital assets, at a high level, and over time, the FAIR guiding principles have been re-interpreted or extended to include the software, tools, algorithms, and workflows that produce data. FAIR principles are now being adapted in the context of AI models and datasets. Here, we present the perspectives, vision, and experiences of researchers from different countries, disciplines, and backgrounds who are leading the definition and adoption of FAIR principles in their communities of practice, and discuss outcomes that may result from pursuing and incentivizing FAIR AI research. The material for this report builds on the FAIR for AI Workshop held at Argonne National Laboratory on June 7, 2022.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Integrated System Planning: Emerging Software Requirements in the Power Industry

Power system planning software remains fragmented across organizational boundaries, with specialized tools for capacity expansion, production cost modeling, power flow, and dynamic analysis operating on incompatible data models and assumptions. This article argues that the fragmentation is not merely a technical problem but a predictable consequence of Conway's law: software architectures mirror the departmental structures within which they are developed. Regulatory milestones like Federal Energy Regulatory Commission (FERC) Order 888 formalized these divisions, but the roots trace back to the distinct engineering disciplines-mechanical, chemical, and electrical-that staffed generation and transmission planning departments in vertically integrated utilities. As the industry moves toward integrated system planning (ISP) that coordinates generation, transmission, and distribution investment decisions, the software ecosystem must evolve accordingly. We identify five categories of software requirements to enable this transition: coherent data inputs decoupled from individual applications, unified and extensible data schemas, modular component representations that support multiple abstraction levels, lifecycle management of planning datasets, and well-defined application programming interface (API) contracts that separate data exchange from algorithmic control. We examine how these requirements interact with three common workflow patterns-serial gate clearing, sequential multiapplication, and convergence oriented-and discuss the interface design principles each demands. We then outline a vision for platform-based planning architectures where specialized analytical services compose through standardized interfaces and where artificial intelligence (AI)/machine learning (ML) tools augment decision support within a disciplined software infrastructure. The practices proposed here offer a path from today's siloed tool collections toward collaborative planning ecosystems capable of handling the complexity of modern power system transformation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Revolutionizing Waste Management: AI-Powered Real-Time Characterization for Efficient Handling of Non-Recyclable Municipal Solid Waste

According to EPA, -300 MM tons of municipal solid waste (MSW) was available in the US as of 2018. Of that total material, nearly 50% was landfilled resulting in a significant loss for the potential to convert its energy value into cost effective and sustainable biofuels. Redirecting this material away from the landfill and into conversion ready feedstock for energy generation can directly address DOE's selling price < $2.50/GGE while securing the US national energy independence [1]. However, the paramount challenges in any rational fuel conversion strategy are understanding the chemical makeup, quality and associated calorific value of the MSW. Understanding these parameters is critical in achieving any acceptable fuel conversion and requires rapid characterization followed by accurate separation technologies. Therefore, we are proposing to address the rapid characterization by building a non-invasive, rapid, and highly accurate Artificial Intelligence (AI)-enabled spectrometric/optical approach augmented with multi-sensory information for advanced characterization of domestic heterogeneous MSW. North Carolina State University (NCSU) and the National Renewable Energy Laboratory (NREL), in partnership with strong support from the Town of Cary and IBM, Inc., will closely work together to implement this ground-breaking technology for the effective characterization of MSW for sustainable and affordable production of conversion-ready feedstocks, while solving the environment issue of planetary proportions.

artificial intelligence↗
DOE OSTI2026

Illuminating the Material World: Autonomous Microscopy to Understand Order, Disorder, and Everything In Between

Artificial intelligence (AI) holds immense promise for revolutionizing microscopy, yet its widespread adoption has been hindered by challenges ranging from user inexperience to limited model transferability and difficulties in operationalizing machine learning. This presentation showcases our approach to developing practical autonomy for materials discovery, aiming to accelerate the integration of AI into everyday microscopy workflows. As shown in Fig. 1, I will focus on three key areas: understanding order-disorder transitions, quantifying point defects, and achieving truly device-scale microscopy. First, I will demonstrate the power of multi-modal knowledge graphs for integrating diverse microscopy data. By combining imaging, spectroscopy, and diffraction data, these graphs provide a holistic view of material behavior, capturing the intricate relationships between different modalities [1,2]. I will present a case study on how these models illuminate the structural and chemical changes associated with irradiation in oxide thin films, revealing critical insights for designing materials for extreme environments like spaceflight and nuclear energy. Specifically, I will show how multi-modal analysis clarifies the evolution of order-disorder transitions under irradiation, a key factor influencing material performance in these applications. Next, I will address the challenge of quantifying point defects in 2D materials. We demonstrate the application of computer vision and transfer learning to accurately identify and classify various defect types, such as vacancies and substitutional atoms, and to quantify their concentrations. This information is crucial for understanding and tailoring the properties of 2D materials for applications in electronics, optoelectronics, and catalysis. For example, I will show how our models can characterize the topological distribution of point defects in MXene transition metal carbides, providing valuable insights for optimizing their performance in energy storage and separation science. Finally, I will discuss our progress toward autonomous device-scale microscopy [3,4]. We are fundamentally redesigning electron microscopes around the principles of machine reasoning, enabling automation beyond basic tasks like sample navigation and data acquisition to include sophisticated experimental design. This approach paves the way for truly reproducible and massively scaled analysis campaigns. I will emphasize the importance of autonomous microscopy platforms for high-throughput materials discovery and characterization, facilitating the rapid screening of materials for a broad range of applications and accelerating the development of next-generation technologies.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Integrated Research Infrastructure Architecture Blueprint Activity (Final Report 2023)

The complexity of scientific pursuits is increasing rapidly with aspects that require dynamic integration of experiment, observation, theory, modeling, simulation, visualization, machine learning (ML), artificial intelligence (AI), and analysis. Research projects across the Department of Energy (DOE) are increasingly data and compute intensive. Innovative research teams are accelerating the pace of discovery by using high-performance computational and data tools in their research workflows and leveraging multiple research infrastructures. Additionally, several recent high-level U.S. government reports underscore the necessity of a new advanced computing ecosystem for international competitiveness and national security. International competitors are moving forward with major research infrastructure integration efforts that seek to capture a competitive advantage in the global innovation race. Owing to its unparalleled constellation of world-class experimental and observational facilities and high-performance and extreme-scale computational, data, and networking infrastructure, DOE is positioned to be a global leader in this new era of integrated science. However, this new integration paradigm will demand continuing evolution to ensure the U.S. remains a global leader in research and innovation. The DOE Office of Science (SC) has seized on the strategic importance of integration and has adopted a vision for Integrated Research Infrastructure (IRI): To empower researchers to meld DOE’s world-class research tools, infrastructure, and user facilities seamlessly and securely in novel ways to radically accelerate discovery and innovation. To respond to the evolving computational requirements of research and the competitive international innovation landscape, experimental facilities could be connected with high performance computing resources for near real-time analysis, and resources should be provided for merging enormous and diverse data for AI/ML techniques and analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Designing Future Energy Systems with Generative AI

Energy systems are experiencing various changes that impact the distribution, use, and reliability of energy. Local utilities and municipalities must respond and adapt to these changes, moving towards a future energy system with modernized infrastructure and other targeted investments and policy decisions. However, planning for and enacting these advancements requires significant effort from experts and engineers to develop strategies that ensure a reliable and secure energy future. This includes characterizing the current energy infrastructure, identifying areas for development, and engaging with local community members. Emerging generative artificial intelligence techniques can alleviate pain points and help support the development of the next generation of energy systems. Here, in this article, we highlight on-going generative AI work in the areas of atmospheric modeling, building energy management, and distribution network design, and we propose a vision for the role of generative AI that considers opportunities and identifies challenges inherent to this technology.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

Dataset for Leveraging CryoEM and AI-Driven Morphological Feature Analysis for Insights on Bacterial Structures

This repository hosts an AI-assisted image segmentation and analysis pipeline for Pantoea sp. YR343 cryo-electron microscopy (cryoEM) datasets. The workflow automates membrane thickness measurements, flagella detection, and field-of-view (FOV) screening from low-dose, high-resolution cryoEM micrographs eliminating the need for slow manual annotation. By integrating deep-learning based segmentation (YOLOv11) with quantitative post-processing, this toolkit provides a scalable and reproducible way to study bacterial morphology under hydrated, near-native conditions. The GitHub repository for AI-based tools for cryoEM bacteria ultrastructures can be found here: https://github.com/Sireesiru/Cryo-EM-Ultrastructures/tree/main

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

ESMs Latent Space Exploration for Uncertainty Quantification and Spatiotemporal Downscaling

This final report for DOE Award DE-SC0023044 presents advances in two key areas of climate modeling: (1) representative climate model selection and (2) Earth System Model (ESM) downscaling using hybrid AI methods. The first section introduces a reordered, three-stage workflow to select representative GCM runs that more effectively balance historical skill with ensemble spread, validated across Texas, Bihar, and New York. The second section introduces two novel super-resolution frameworks, ViSIR and ViFOR, that integrate Vision Transformers with sinusoidal and Fourier-based implicit neural representations. These models achieve state-of-the-art reconstruction accuracy for ESM variables including surface temperature and heat fluxes. The report includes detailed methodology, benchmarks, and results, demonstrating significant gains in uncertainty quantification, spatial fidelity, and scalability for climate-impact studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

White Paper: Scalable Digital Twin Capabilities for Aging and Surveillance of Engineered Systems

This white paper presents a multi-year initiative to develop practical, secure, and scalable digital twin capabilities for engineered systems in aging and surveillance contexts—an approach pioneered at the National Nuclear Security Administration (NNSA) Lawrence Livermore National Laboratory (LLNL) that maps directly onto the needs and ambitions of the Navy for ship- and fleet-level digital twins. LLNL’s work in building part- and process-level digital twins for advanced manufacturing, with a vision to scale up to entire factory floors and, ultimately, enterprise-wide digital twins, offers an adaptable pathway for the Navy as it seeks to modernize lifecycle management, readiness, and predictive maintenance across ships and fleets. For our application, we integrate physics-based modeling with automated data ingestion, processing, and AI-driven calibration, creating hybrid models that are both interpretable and data responsive. We modernized legacy workflows, established centralized data infrastructure, automated experimental pipelines, and demonstrated end-to-end coupling of accelerated aging data with finite element simulations via optimization and surrogate modeling. The result is a generalizable framework that supports part-level digital twins today and lays the groundwork for future system-level twins suitable for Navy applications.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Laboratory Directed Research and Development Program Activities (LDRD 2021 Annual Report)

Each year, Brookhaven National Laboratory (BNL) is required to provide a report of its completed Laboratory Directed Research and Development Program (LDRD) projects to the Department of Energy (DOE) Office of Scientific and Technical Information in accordance with DOE Order 413.2C Chg1 (MinChg) dated August 2, 2018. This report provides a detailed look at the scientific and technical activities for each of the LDRD projects funded by BNL in FY 2021, in fulfillment of that requirement. In FY 2021, the BNL LDRD Program funded 68 projects, 24 of which were new starts, at a total cost of $17.1M. The investments that BNL makes in its LDRD program support the Laboratory’s strategic goals. BNL has identified six scientific initiatives that define the Laboratory’s scientific future and that will enable it to realize its overall vision. This requires simultaneous excellence in all aspects of BNL’s work – from science and operations, to external partnerships with the local, state, and national communities, and beyond. This is enabled by safe, efficient, and secure operations; by an unwavering commitment to a diverse, equitable, and inclusive environment, including workforce development, both with staff and reaching out to the community; and by a strong focus on renewed infrastructure. The six scientific initiatives are: (1) Nuclear Physics: The Electron Ion Collider, (2) Clean Energy and Climate, (3) Quantum Information Science and Technology, (4) Discovery Science Driven by the Human-AI-Facility Integration, (5) High Energy Physics: Building for Discovery, and (6) Accelerating Isotope Production: Ensuring the Nation’s Supply is Secure. The funded projects support BNL’s six scientific initiatives and priority programs as well as new areas of research and competencies at the Laboratory that are consistent with the Laboratory’s vision and mission. In total, these LDRD investments supported 80 postdoctoral researchers and graduate students in whole or in part and resulted in 136 publications and 3 awards. This Program Activities Report represents the future of BNL science; it is an impressive body of exploratory work that investigates many scientific and technical directions in support of the DOE and BNL missions.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2020

Measuring Cities with Software-Defined Sensors

The Chicago Array of Things (AoT) project, funded by the US National Science Foundation, created an experimental, urban-scale measurement capability to support diverse scientific studies. Initially conceived as a traditional sensor network, collaborations with many science communities guided the project to design a system that is remotely programmable to implement Artificial Intelligence (AI) within the devices-at the “edge” of the network-as a means for measuring urban factors that heretofore had only been possible with human observers, such as human behavior including social interaction. The concept of “software-defined sensors” emerged from these design discussions, opening new possibilities, such as stronger privacy protections and autonomous, adaptive measurements triggered by events or conditions. We provide examples of current and planned social and behavioral science investigations uniquely enabled by software-defined sensors as part of the SAGE project, an expanded follow-on effort that includes AoT.

97 MATHEMATICS AND COMPUTING↗