Search NASA⌕ Search

SEARCH · Search NASA

Results for “representation learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

From PINNs to PIKANs: recent advances in physics-informed machine learning

Physics-Informed Neural Networks (PINNs) have emerged as a key tool in Scientific Machine Learning since their introduction in 2017, enabling the efficient solution of ordinary and partial differential equations using sparse measurements. Over the past few years, significant advancements have been made in the training and optimization of PINNs, covering aspects such as network architectures, adaptive refinement, domain decomposition, and the use of adaptive weights and activation functions. A notable recent development is the Physics-Informed Kolmogorov-Arnold Networks (PIKANS), which leverage a representation model originally proposed by Kolmogorov in 1957, offering a promising alternative to traditional PINNs. In this review, we provide a comprehensive overview of the latest advancements in PINNs, focusing on improvements in network design, feature expansion, optimization techniques, uncertainty quantification, and theoretical insights. We also survey key applications across a range of fields, including biomedicine, fluid and solid mechanics, geophysics, dynamical systems, heat transfer, chemical engineering, and beyond. Lastly, we review computational frameworks and software tools developed by both academia and industry to support PINN research and applications.

Kolmogorov-Arnold networks↗
DOE OSTI2024

Learning from metastable symmetric-tilt grain boundaries using physics-based descriptors

Grain boundaries (GBs) govern critical properties of polycrystalline materials. Although significant advancements have been made in characterizing minimum energy and ordered GBs, real GBs are seldom found in such well-defined states. This diversity of atomic arrangements in metastable states makes it challenging to establish structure-property relationships with physical insights. Here, to address this challenge, we use data-driven methods to explore these relationships and examine the underlying physics. In this study, we utilize a large atomistic database (~5000) of minimum energy and metastable states of symmetric-tilt copper GBs, combined with physically motivated local atomic environment (LAE) descriptors [strain functional descriptors (SFDs)], to predict GB properties and gain physical insights. Our regression models exhibit robust predictive capabilities using only 19 descriptors, generalizing to atomic environments in nanocrystals. A significant highlight of our work is the integration of an unsupervised method with SFDs to elucidate LAEs at GBs and their role in determining properties. The model, trained on these minimum energy and metastable GBs using SFDs, predicts the properties of unseen nanocrystals with good accuracy. Our research underscores the role of a physics-based representation of LAEs and the efficacy of data-driven methods in establishing GB structure-property relationships.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Automating the Analysis of Large Language Models Responses through Zero-Shot Question Answering

Recent advancements in Large Language Models (LLMs) have shown significant potential in various applications, yet their evaluation, particularly in zero-shot question answering scenarios, remains a challenging task. In this study, our objective was to explore precision metrics for Large Language Models (LLM) and design and implement a software pipeline to automatically evaluate LLMs' outputs under zero-shot question answering. Zero-shot question answering involves a model providing answers to questions about topics it hasn't seen during training. It leverages the principles of zero-shot learning by relying on semantic understanding and generalization from related knowledge. The data used was metadata from medical databases on congenital heart disease. We explored eleven LLM metrics and selected three for our evaluation: BLEU, BERTScore, and MoverScore. BLEU calculates a score based on the overlap of n-grams (contiguous sequences of n items, typically words) between the machine-generated translation and the reference translations. Higher BLEU scores indicate better correspondence between the machine-generated and human-generated translations. BERTScore is a metric used to evaluate the quality of machine-generated text by measuring the similarity of token embeddings produced by BERT (Bidirectional Encoder Representations from Transformers) between the generated text and reference text. MoverScore is a metric that quantifies the dissimilarity between the distributions of word embeddings from machine-generated text and reference text, emphasizing semantic similarity over exact token overlap. We also introduced HBKI, a composite metric summarizing these approaches. We tested five models —GPT-3, Llama-2, Gemini 1.5 Pro, Solar 10.7B, and Mixtral-8x7b. Our software pipeline, designed and implemented using Object-Oriented Programming principles, allows users to customize the selection and extraction of features for topics of interest in their own research. Our results show that MoverScore delivered the most precise evaluation of the LLM's outputs, while Mixtral-8x7b achieved the best overall performance in extracting metadata from the databases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Harnessing distributed GPU computing for generalizable graph convolutional networks in power grid reliability assessments

Although machine learning (ML) has emerged as a powerful tool for rapidly assessing grid contingencies, prior studies have largely considered a static grid topology in their analyses. This limits their application, since they need to be re-trained for every new topology. Here, this paper explores the development of generalizable graph convolutional network (GCN) models by pre-training them across a range of grid topologies and contingency types. We found that a GCN model with auto-regressive moving average (ARMA) layers with a line graph representation of the grid offered the best predictive performance in predicting voltage magnitudes (VM) and voltage angles (VA). We introduced the concept of phantom nodes to consider disparate grid topologies with a varying number of nodes and lines. For pre-training the GCN ARMA model across a variety of topologies, distributed graphics processing unit (GPU) computing afforded us significant training scalability. The predictive performance of this model on grid topologies that were part of the training data is substantially better than the direct current (DC) approximation. Although direct application of the pre-trained model to topologies that are not part of the grid is not particularly satisfactory, fine-tuning with small amounts of data from a specific topology of interest significantly improves predictive performance. In general, this paper highlights the feasibility of training large-scale GNN models to assess the reliability of power grids by considering a wide variety of grid topologies and contingency types. With the advent of foundational models in ML and the exponential increase in GPU computing clusters, generalizable ML models will significantly enhance how utilities manage power systems and make decisions in real-time or near-real-time.

24 - POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Generative Models for Crystalline Materials

Understanding structure-property relationships in materials is fundamental in condensed matter physics and materials science. Over the past few years, machine learning (ML) has emerged as a powerful tool for advancing this understanding and accelerating materials discovery. Early ML approaches primarily focused on constructing and screening large material spaces to identify promising candidates for various applications. More recently, research efforts have increasingly shifted toward generating crystal structures using end-to-end generative models. This review analyzes the current state of generative modeling for crystal structure prediction and de novo generation. It examines crystal representations, outlines the generative models used to design crystal structures, and evaluates their respective strengths and limitations. Furthermore, the review highlights experimental considerations for evaluating generated structures and provides recommendations for suitable existing software tools. Emerging topics, such as modeling disorder and defects, integration in advanced characterization, incorporating synthetic feasibility constraints, and model explainability are explored. Ultimately, this work aims to inform both experimental scientists looking to adapt suitable ML models to their specific circumstances and ML specialists seeking to understand the unique challenges related to inverse materials design and discovery.

Metni, Houssam [Karlsruhe Inst. of Technology (KIT↗
DOE OSTI2025

Unsupervised discovery of extreme weather events using universal representations of emergent organization

Spontaneous self-organization is ubiquitous in systems far from thermodynamic equilibrium. While organized structures that emerge dominate transport properties, universal representations that identify and describe these key objects remain elusive. Here, we introduce a theoretically grounded framework for describing emergent organization that, via data-driven algorithms, is constructive in practice. Its building blocks are spacetime lightcones that embody how information propagates across a system through local interactions. We show that predictive equivalence classes of lightcones—local causal states—capture organized behaviors in complex spatiotemporal systems. Employing an unsupervised physics-informed machine learning algorithm and a high-performance computing implementation, we demonstrate automatically discovering organized structures in two real-world domain science problems. We show that local causal states identify vortices and track their power-law decay behavior in two-dimensional fluid turbulence. We then show how to detect and track familiar extreme weather events—hurricanes and atmospheric rivers—and discover other novel structures associated with precipitation extremes in high-resolution climate data at the grid-cell level.

Rupe, Adam [Pacific Northwest National Laboratory ↗
NASA NTRS1988

PDA: A coupling of knowledge and memory for case-based reasoning

Problem solving in most domains requires reference to past knowledge and experience whether such knowledge is represented as rules, decision trees, networks or any variant of attributed graphs. Regardless of the representational form employed, designers of expert systems rarely make a distinction between the static and dynamic aspects of the system's knowledge base. The current paper clearly distinguishes between knowledge-based and memory-based reasoning where the former in its most pure sense is characterized by a static knowledge based resulting in a relatively brittle expert system while the latter is dynamic and analogous to the functions of human memory which learns from experience. The paper discusses the design of an advisory system which combines a knowledge base consisting of domain vocabulary and default dependencies between concepts with a dynamic conceptual memory which stores experimental knowledge in the form of cases. The case memory organizes past experience in the form of MOPs (memory organization packets) and sub-MOPs. Each MOP consists of a context frame and a set of indices. The context frame contains information about the features (norms) common to all the events and sub-MOPs indexed under it.

Bharwani, S.↗
NASA NTRS2024

Digital Lunar Exploration Sites (DLES) Terrain Crafting

Humans will soon be returning to the surface of the Moon with NASA’s Artemis program. The Artemis program is an international collaboration that will consist of a complex series of space systems and missions to explore the lunar surface and pave the way for the future exploration of Mars. NASA and its partners rely heavily on simulation for lighting and navigation studies as well as training astronauts, flight controllers, and mission support staff. The NASA Exploration Systems Simulations (NExSyS) team in the Simulation and Graphics Branch (ER7) in the Engineering Directorate at NASA’s Johnson Space Center has built up many simulation products to support this effort, one of which is the Digital Lunar Exploration Sites (DLES). DLES is a collection of products used to simulate and render the lunar surface in a digital environment. We discussed and presented an overview of the DLES products at the 2022 IEEE Aerospace Conference in Big Sky, MT with a paper titled "Digital Lunar Exploration Sites". This “DLES Terrain Crafting” paper will expand on the information previously provided in “DLES” paper and dive deeper into the details of the terrain crafting process and the toolsets used to support this task. The best digital data currently available of the lunar surface is provided by the Lunar Reconnaissance Orbiter (LRO). Its Lunar Orbiter Laser Altimeter (LOLA) achieves an impressive resolution of 5m per pixel at the Lunar South Pole (LSP) and can generate datasets covering a large continuous region near the LSP. There are a few additional methods, such as Shape from Shading which can infer higher resolution data (up to 1m per pixel) from the LRO Narrow Angle Camera (NAC) images. However, surface-based simulations require higher-resolution data, and this paper will discuss the process of enhancing the terrain to meet that need. The process begins with capturing statistical data of craters in the regions of interest using images provided by the LRO NAC. This data is then used to scatter artificial features which are not captured in the truth data, resulting in an enhanced DEM with a much higher resolution of 20cm per pixel. Many tools were built up to assist in the creation of these artificial Digital Elevation Models (DEM), which this paper will discuss in detail. DEMs themselves are a very powerful representation of a planetary surface, and many operations and tools can utilize the data they contain. This paper includes a description of the rendering of the lunar surface in a graphics engine, generation of contact patches to simulate tire to ground interaction, and ray tracing utilities to model Line of Sight (LOS) interactions with the terrain. This paper will also explore some new tool sets currently under development which aim to utilize Machine Learning (ML) to assist in the identification of craters from LRO NAC imagery. While this is not a novel idea, the NExSyS team is developing a unique approach which may result in more robust identification of crater characteristics.

Artemis↗
NASA NTRS1990

Designing a training tool for imaging mental models

The training process can be conceptualized as the student acquiring an evolutionary sequence of classification-problem solving mental models. For example a physician learns (1) classification systems for patient symptoms, diagnostic procedures, diseases, and therapeutic interventions and (2) interrelationships among these classifications (e.g., how to use diagnostic procedures to collect data about a patient's symptoms in order to identify the disease so that therapeutic measures can be taken. This project developed functional specifications for a computer-based tool, Mental Link, that allows the evaluative imaging of such mental models. The fundamental design approach underlying this representational medium is traversal of virtual cognition space. Typically intangible cognitive entities and links among them are visible as a three-dimensional web that represents a knowledge structure. The tool has a high degree of flexibility and customizability to allow extension to other types of uses, such a front-end to an intelligent tutoring system, knowledge base, hypermedia system, or semantic network.

Dede, Christopher J.↗
NASA NTRS1992

Spike: AI scheduling for Hubble Space Telescope after 18 months of orbital operations

This paper is a progress report on the Spike scheduling system, developed by the Space Telescope Science Institute for long-term scheduling of Hubble Space Telescope (HST) observations. Spike is an activity-based scheduler which exploits artificial intelligence (AI) techniques for constraint representation and for scheduling search. The system has been in operational use since shortly after HST launch in April 1990. Spike was adopted for several other satellite scheduling problems; of particular interest was the demonstration that the Spike framework is sufficiently flexible to handle both long-term and short-term scheduling, on timescales of years down to minutes or less. We describe the recent progress made in scheduling search techniques, the lessons learned from early HST operations, and the application of Spike to other problem domains. We also describe plans for the future evolution of the system.

Johnston, Mark D.↗
NASA NTRS2022

Shape Servoing of Deformable Objects using Adaptive Deformation Model Estimation

In this paper, we propose an adaptive shape servoing method to deform a soft object into a desired 3-D shape. The high dimensional representation and the unknown deformation properties of the soft object pose a challenge to actively manipulate its shape. To address this issue, we develop a method to compute the deformation Jacobian matrix in real-time. The Jacobian is estimated using a set of basis functions and its corresponding parameters to capture the dynamics of the system and relate the applied input motion to changes in the soft object's shape. An integral concurrent learning (ICL) based adaptive update law is derived using Lyapunov analysis to estimate the deformation parameters and prove its convergence. A physics-based simulation is used to validate the proposed method and controller by performing manipulation tasks with different desired configurations. The performance is compared with a standard gradient update law to demonstrate the accuracy and robustness of our approach.

Vrithik Raj Guthikonda↗
DOE OSTI2024

LossLens: Diagnostics for Machine Learning Through Loss Landscape Visual Analytics

Modern machine learning often relies on optimizing a neural network's parameters using a loss function to learn complex features. Beyond training, examining the loss function with respect to a network's parameters (i.e., as a loss landscape) can reveal insights into the architecture and learning process. While the local structure of the loss landscape surrounding an individual solution can be characterized using a variety of approaches, the global structure of a loss landscape, which includes potentially many local minima corresponding to different solutions, remains far more difficult to conceptualize and visualize. To address this difficulty, we introduce LossLens, a visual analytics framework that explores loss landscapes at multiple scales. LossLens integrates metrics from global and local scales into a comprehensive visual representation, enhancing model diagnostics. Here we demonstrate LossLens through two case studies: visualizing how residual connections influence a ResNet-20, and visualizing how physical parameters influence a physics-informed neural network (PINN) solving a simple convection problem.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2017

Neural Predictors of Visuomotor Adaptation Rate and Multi-Day Savings

Recent studies of sensorimotor adaptation have found that individual differences in task-based functional brain activation are associated with the rate of adaptation and savings at subsequent sessions. However, few studies to date have investigated offline neural predictors of adaptation and multi-day savings. In the present study, we explore whether individual differences in the rate of visuomotor adaptation and multi-day savings are associated with differences in resting state functional connectivity and gray matter volume. Thirty-four participants performed a manual adaptation task during two separate test sessions, on average 9 days apart. We found that resting state functional connectivity strength between sensorimotor, anterior cingulate, and temporoparietal areas of the brain was a significant predictor of adaptation rate during the early, cognitive phase of practice. In contrast, default mode network functional connectivity strength was found to predict late adaptation rate and savings on day two, which suggests that these behaviors may rely on overlapping processes. We also found that gray matter volume in temporoparietal and occipital regions was a significant predictor of early learning, whereas gray matter volume in superior posterior regions of the cerebellum was a significant predictor of late adaptation. The results from this study suggest that offline neural predictors of early adaptation facilitate the cognitive mechanisms of sensorimotor adaptation, with support from by the involvement of temporoparietal and cingulate networks. In contrast, the neural predictors of late adaptation and savings, including the default mode network and the cerebellum, likely support the storage and modification of newly acquired sensorimotor representations. These findings provide novel insights into the neural processes associated with individual differences in sensorimotor adaptation.

Cassady, Kaitlin↗
DOE OSTI2025

Advancing the Representation of Human Actions in Large‐Scale Hydrological Models: Challenges and Future Research Directions

Characterizing the impact of human actions on terrestrial water fluxes and storages at multi-basin, continental, and global scales has long been on the agenda of scientists engaged in climate science, hydrology, and water resources systems analysis. This need has resulted in a variety of modeling efforts focused on the representation of water infrastructure operations. Yet, the representation of human-water interactions in large-scale hydrological models is still relatively crude, fragmented across models, and often achieved at coarse resolutions (~10–100 km) that cannot capture local water management decisions. In this commentary, we argue that the concomitance of four drivers and innovations is poised to change the status quo: “hyper-resolution” hydrological models (~0.1–1 km), multi-sector modeling, satellite missions able to monitor the outcome of human actions, and machine learning are creating a fertile environment for human-water research to flourish. We then outline four challenges that chart future research in hydrological modeling: (a) creating hyper-resolution global data sets of water management practices, (b) improving the characterization of anthropogenic interventions on water quantity, stream temperature, and sediment transport, (c) improving model calibration and diagnostic evaluation, and (d) reducing the computational requirements associated with the successful exploration of these challenges. Overcoming them will require addressing modeling, computational, and data development needs that cut across the hydrology community, thereby requiring a major communal effort.

catchment hydrology↗
NASA NTRS1993

Program Helps In Analysis Of Failures

Failure Environment Analysis Tool (FEAT) computer program developed to enable people to see and better understand effects of failures in system. User selects failures from either engineering schematic diagrams or digraph-model graphics, and effects or potential causes of failures highlighted in color on same schematic-diagram or digraph representation. Uses digraph models to answer two questions: What will happen to system if set of failure events occurs? and What are possible causes of set of selected failures? Helps design reviewers understand exactly what redundancies built into system and where there is need to protect weak parts of system or remove them by redesign. Program also useful in operations, where it helps identify causes of failure after they occur. FEAT reduces costs of evaluation of designs, training, and learning how failures propagate through system. Written using Macintosh Programmers Workshop C v3.1. Can be linked with CLIPS 5.0 (MSC-21927, available from COSMIC).

Stevenson, R. W.↗
NASA NTRSDate not supplied

Reanalysis Activities at the NASA Global Modeling and Assimilation Office

This talk presents an overview of recent reanalysis activities at the NASA Global Modeling and Assimilation Office (GMAO) as part of a multi-faceted strategy towards an Integrated Earth System retrospective analysis, coupling components of the atmosphere, ocean, chemistry, land, and ice. While elements of the atmosphere-ocean coupled Goddard Earth Observing System (GEOS) model and data assimilation are being actively developed, a suite of reanalysis products is designed to provide further understanding of key aspects of Earth system coupling in a reanalysis context: The baseline atmospheric reanalysis, the GEOS Retrospective analysis for the early 21st Century (GEOS-R21C), features recent advances in the GEOS model and data assimilation, and targets the NASA’s Earth Observing System EOS and post-EOS satellite observations; GEOS-IT, a user-tailored low-resolution atmospheric reanalysis, serves as a second baseline to the NASA Instrument Teams for validation and calibration and drives a one-way coupled ocean reanalysis, GEOSIT-Ocean; PolarMERRA, a high-resolution downscaled product for the polar regions, focuses on improving the representation of polar atmospheric processes with an assessment of current cryospheric biases, and targeted improvements to surface sea ice and glacier conditions; Finally, R21C-Chem, an off-line atmospheric chemistry and composition reanalysis, includes both tropospheric and stratospheric trace gases. The diversity of these reanalysis activities presents unique opportunities for collaborations cross-teams/institutions, with new commercial data partners, and with end-user groups. This talk will discuss these opportunities and explore leveraging the lessons learned along the way on key drivers in Earth system interactions as we converge towards the next generation of the Modern-Era Retrospective analysis for Research and Applications (MERRA) suite.

Amal El Akkraoui↗
DOE OSTI2024

Geothermal Power Systems Analysis: Outcome of Industry Stakeholders Workshop: Preprint

Geothermal cost and performance evaluation implemented via technoeconomic assessment (TEA) modeling is critical for the Department of Energy (DOE) and other geothermal industry stakeholders in assessing the current state of geothermal technologies and to identify existing hurdles to commercially viable geothermal development. The Geothermal Electricity Technology Evaluation Model (GETEM) is a major TEA tool used in estimating the economic feasibility and levelized cost of energy (LCOE) of conventional hydrothermal systems and enhanced geothermal systems (EGS). Since 2021, GETEM has been transitioning from an intricate spreadsheet model to a user-friendly tool within the System Advisor Model (SAM) developed by the National Renewable Energy Laboratory (NREL). Apart from enabling an expanded visibility of the geothermal model among other renewable resources, having GETEM in SAM has the advantage of simulation automation, better usability, updates tracking, active user inputs/feedback, and extended financial modeling. GETEM is used in developing supply curves for the Annual Technology Baseline (ATB). The ATB data are inputs to the Renewable Energy Potential (reV) and the Regional Energy Deployment System (ReEDS) models. The geothermal module in NREL’s reV model assesses the geothermal energy potential in the conterminous United States by defining the geospatial intersection of geothermal resources with existing grid infrastructure within the constraint of land use characteristics. The ReEDS model is a capacity expansion model used for simulating the long-term build-out and operation of the US generation and transmission system based on current energy costs and policies. To ensure enhanced representation of current industry trends in our model transitions and development, we organized a two-day virtual workshop to elicit geothermal industry stakeholder input and recommendations on our current approaches and assumptions on technoeconomic, resource assessment, and deployment scenarios modeling of geothermal technologies. Participants included developers, operators, investors, regulatory agencies, system modelers, national laboratory researchers, consultants, and other stakeholders. In this workshop, we gained stakeholder insights on current geothermal plant performance (i.e., capacity factors), updated drilling costs and learning curves, and next generation technologies such as closed loop and superhot rock geothermal. Other outcomes from this workshop and its impact on future geothermal development feasibility, resource availability, and capacity expansion studies are compiled and discussed.

Annual Technology Baseline↗
DOE OSTI2024

ReLU, Sparseness, and the Encoding of Optic Flow in Neural Networks

Accurate self-motion estimation is critical for various navigational tasks in mobile robotics. Optic flow provides a means to estimate self-motion using a camera sensor and is particularly valuable in GPS- and radio-denied environments. The present study investigates the influence of different activation functions—ReLU, leaky ReLU, GELU, and Mish—on the accuracy, robustness, and encoding properties of convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) trained to estimate self-motion from optic flow. Our results demonstrate that networks with ReLU and leaky ReLU activation functions not only achieved superior accuracy in self-motion estimation from novel optic flow patterns but also exhibited greater robustness under challenging conditions. The advantages offered by ReLU and leaky ReLU may stem from their ability to induce sparser representations than GELU and Mish do. Our work characterizes the encoding of optic flow in neural networks and highlights how the sparseness induced by ReLU may enhance robust and accurate self-motion estimation from optic flow.

97 MATHEMATICS AND COMPUTING↗