Search NASA⌕ Search

SEARCH · Search NASA

Results for “foundation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Elastic Solutions to 2D Plane Strain Problems: Nonlinear Contact and Settlement Analysis for Shallow Foundations

The classical Neumann boundary value problem of an isotropic, homogeneous elastic half-plane under plane strain conditions is readdressed as the limiting case of the fully three-dimensional problem. Analytical solutions of the stress and strain tensors are obtained by taking the limit from known three-dimensional solutions. It is shown that the displacement fields for the plane strain problem are not well defined. A small number of simple expressions are developed, which provide a general solution for linearly-varying traction over arbitrary regions on the boundary. A simple, efficient, and rapidly convergent algorithm is developed which uses these solutions as analytic elements and provides a solution approach to the general boundary value problem. The method is verified against known solutions for Hertzian contact between parallel cylinders. Two numerical examples are presented for the analysis of shallow foundation systems. In the first, the boundary conditions are informed by analytical elastoplastic calculations and a strain influence analysis is performed and compared with the Schmertmann method. Subsequently, empirical laboratory contact traction distributions measured by Bauer et al., in both the normal and tangential directions are employed as boundary conditions for an analysis of the underlying stress field.

42 ENGINEERING↗
DOE OSTI2024

From RNNs to Foundation Models: An Empirical Study on Commercial Building Energy Consumption

Accurate short-term energy consumption forecasting for commercial buildings is crucial for smart grid operations. While smart meters and deep learning models enable forecasting using past data from multiple buildings, data heterogeneity from diverse buildings can reduce model performance. The impact of increasing dataset heterogeneity in time series forecasting, while keeping size and model constant, is understudied. We tackle this issue using the ComStock dataset, which provides synthetic energy consumption data for U.S. commercial buildings. Two curated subsets, identical in size and region but differing in building type diversity, are used to assess the performance of various time series forecasting models, including finetuned open-source foundation models (FMs). The results show that dataset heterogeneity and model architecture have a greater impact on post-training forecasting performance than the parameter count. Moreover, despite the higher computational cost, finetuned FMs demonstrate competitive performance compared to base models trained from scratch.

commercial buildings↗
DOE OSTI2026

Multi-task Parallelism for Robust Pre-training of Graph Foundation Models on Multi-source, Multi-fidelity Atomistic Modeling Data

Graph foundation models using graph neural networks promise sustainable, efficient atomistic modeling. To tackle challenges of processing multi-source, multi-fidelity data during pre-training, recent studies employ multi-task learning, in which shared message passing layers initially process input atomistic structures regardless of source, then route them to multiple decoding heads that predict data-specific outputs. This approach stabilizes pre-training and enhances a model’s transferability to unexplored chemical regions. Preliminary results on approximately four million structures are encouraging, yet questions remain about generalizability to larger, more diverse datasets and scalability on supercomputers. We propose a multi-task parallelism method that distributes each head across computing resources with GPU acceleration. Implemented in the open-source HydraGNN architecture, our method was trained on over 24 million structures from five datasets and tested on the Perlmutter, Aurora, and Frontier supercomputers, demonstrating efficient scaling on all three highly heterogeneous super-computing architectures.

Lupo Pasini, Massimiliano [ORNL] (ORCID:0000000249↗
DOE OSTI2023

Editorial special issue: Advancing foundational sun-induced chlorophyll fluorescence science

The first research on the possibilities offered by chlorophyll-a fluorescence to track the daily course of CO 2 assimilation by leaves dates back to the last century (Kautsky and Hirsch, 1931). In the second half of the XXth Century, the development of the field of active fluorescence, which relies mainly on the pulse amplitude modulation (PAM) fluorimetry technique, allowed the unraveling of the relationship between the yield of chlorophyll-a fluorescence and photochemistry (linear electron transport), which is modulated by a third process known as non-photochemical quenching (Genty et al., 1989). Since then, active fluorescence has been regularly used in ecophysiology, forestry, and crop sciences to understand the plant response to stress (Schreiber, 2004). Since the beginning of the 2000s, the development of portable field spectrometers allows for measuring passive sun-induced fluorescence (SIF) at strong solar or telluric absorption features, which do not rely on the use of artificial excitation light (e.g., Meroni and Colombo, 2006; Meroni et al., 2009; Perez-Priego et al., 2005). Since then, the field has rapidly evolved, and now fluorescence is measurable with automated field spectrometers in the field (Grossmann et al., 2018; Gu et al., 2019b; Rossini et al., 2010), airborne platforms (Rascher et al., 2015; Zarco-Tejada et al., 2000), and satellites (Guanter et al., 2012; Sun et al., 2018). These advances offer the potential to couple ecosystem-scale measurements of CO 2 fluxes and SIF to probe new aspects related to ecosystem structural impacts on SIF andphotosynthesis processes at different time-scales and on different ecosystems (e.g., Damm et al., 2010; Magney et al., 2019; Porcar-Castell et al., 2021). While SIF has been proven as a good proxy of gross primary productivity (GPP), mainly across large spatial and temporal gradients, a series of studies showed that different factors could confound this relationship, namely different canopy structures (e.g., Dechant et al., 2020; Migliavacca et al., 2017), stress conditions (e.g., Martini et al., 2022; Wieneke et al., 2018; Wohlfahrt et al., 2018), nutritional conditions (Cendrero-Mateo et al., 2015; Martini et al., 2019), species-specific differences (e.g., Van Wittenberghe et al., 2013) and light regimes (e.g., Liu and Liu, 2018). At the same time, the modeling of SIF developed rapidly (Gu et al., 2019a; Han et al., 2022; Han et al., 2021; van der Tol et al., 2014; van der Tol et al., 2009) and with an increasing degree of realism, offering the possibility to retrieve important vegetation parameters from remote sensing (Pacheco-Labrador et al., 2019; Verrelst et al., 2015; Verrelst et al., 2016; Zhang et al., 2014). Excellent and comprehensive reviews of the field are by (Mohammed et al., 2019; Porcar-Castell et al., 2021; Porcar-Castell et al., 2014; Sun et al., 2023a; Sun et al., 2023b). Despite the exponential increase in the number of publications in the field, significant progress is still needed. Here, this special issue aims to report foundational SIF science, including theoretical modeling and measurement-based research that is urgently needed to unleash the full potential of SIF for physiological and ecological applications at scales spanning from leaf to globe. The 14 articles collected in this special issue reported on the following specific aspects: first, technical capabilities enabling spectrally resolved SIF observations and their inter-comparability in space and time; second, theoretical developments in SIF-photosynthesis relationships to correctly interpret the signal and extract mechanistic information on vegetation structure and function; third, evaluation of the potential of SIF to track process beyond photosynthesis, such as transpiration; fourth, large scale-applications of SIF observations; and finally, upscaling fluorescence from leaf to canopy scales.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

PDBx/mmCIF Ecosystem: Foundational Semantic Tools for Structural Biology

PDBx/mmCIF, Protein Data Bank Exchange (PDBx) macromolecular Crystallographic Information Framework (mmCIF), has become the data standard for structural biology. With its early roots in the domain of small-molecule crystallography, PDBx/mmCIF provides an extensible data representation that is used for deposition, archiving, remediation, and public dissemination of experimentally determined three-dimensional (3D) structures of biological macromolecules by the Worldwide Protein Data Bank (wwPDB, wwpdb.org). Extensions of PDBx/mmCIF are similarly used for computed structure models by ModelArchive (modelarchive.org), integrative/hybrid structures by PDB-Dev (pdb-dev.wwpdb.org), small angle scattering data by Small Angle Scattering Biological Data Bank SASBDB (sasbdb.org), and for models computed generated with the AlphaFold 2.0 deep learning software suite (alphafold.ebi.ac.uk). Community-driven development of PDBx/mmCIF spans three decades, involving contributions from researchers, software and methods developers in structural sciences, data repository providers, scientific publishers, and professional societies. Having a semantically rich and extensible data framework for representing a wide range of structural biology experimental and computational results, combined with expertly curated 3D biostructure data sets in public repositories, accelerates the pace of scientific discovery. Herein, we describe the architecture of the PDBx/mmCIF data standard, tools used to maintain representations of the data standard, governance, and processes by which data content standards are extended, plus community tools/software libraries available for processing and checking the integrity of PDBx/mmCIF data. Use cases exemplify how the members of the Worldwide Protein Data Bank have used PDBx/mmCIF as the foundation for its pipeline for delivering Findable, Accessible, Interoperable, and Reusable (FAIR) data to many millions of users worldwide.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Dioctyl ether radiolysis under used nuclear fuel reprocessing conditions: Foundational knowledge for the development of sacrificial ligand grafts

Radiation-induced ligand destruction and concomitant degradation product formation is unavoidable under envisioned used nuclear fuel reprocessing conditions, and ultimately limits the recyclability of a given solvent system formulation. With this in mind, a novel approach to ligand innovation for advanced separations processes is in the design of tailored “protomolecules,” which when exposed to ionizing radiation undergo desired chemical transformations, thereby allowing for a reprocessing solvent system to advantageously evolve with absorbed radiation dose. Here we provide foundational knowledge for the time-resolved (electron pulse) and steady-state (cobalt-60 gamma) radiolytic behavior of dioctyl ether, a protomolecule grafting surrogate. Gamma irradiation of single-phase solutions of dioctyl ether (5–100 vol.%) in n-dodecane resulted in the loss of dioctyl ether (G = -0.72 µmol J –1 ) and the formation of octane (G = 0.11 µmol J –1 ) and octanol (G = 0.15 µmol J –1 ) degradation products, the latter of which is a known reprocessing phase modifier and radioprotectant. Further, these radiation-induced changes were attributed to direct radiation effects, for more concentrated dioctyl ether solutions, and indirect radiation effects, predominantly driven by the reaction of dioctyl ether with the dodecane radical cation, for which we report a second-order rate coefficient of k = (1.53 ± 0.05) × 10 10 L mol –1 s –1 . Under typical biphasic extraction (4.0 mol L –1 HNO 3 ) and strip (0.1 mol L –1 HNO 3 ) reprocessing conditions, gamma irradiation of 5 vol.% dioctyl ether solvent systems afforded negligible change in the rate of parent molecule destruction but promoted significant differences in the radiolytic behavior of its degradation products. These differences are attributed to their respective interactions with [dioctyl ether/HNO 3 /H 2 O] and [octanol/HNO 3 /H 2 O] adducts extracted into the organic phase. Overall, these results support the grafting of ether linkages to advanced separations ligands (e.g., modified diglycolamides).

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2024

Learning together: Towards foundation models for machine learning interatomic potentials with meta-learning

Abstract The development of machine learning models has led to an abundance of datasets containing quantum mechanical (QM) calculations for molecular and material systems. However, traditional training methods for machine learning models are unable to leverage the plethora of data available as they require that each dataset be generated using the same QM method. Taking machine learning interatomic potentials (MLIPs) as an example, we show that meta-learning techniques, a recent advancement from the machine learning community, can be used to fit multiple levels of QM theory in the same training process. Meta-learning changes the training procedure to learn a representation that can be easily re-trained to new tasks with small amounts of data. We then demonstrate that meta-learning enables simultaneously training to multiple large organic molecule datasets. As a proof of concept, we examine the performance of a MLIP refit to a small drug-like molecule and show that pre-training potentials to multiple levels of theory with meta-learning improves performance. This difference in performance can be seen both in the reduced error and in the improved smoothness of the potential energy surface produced. We therefore show that meta-learning can utilize existing datasets with inconsistent QM levels of theory to produce models that are better at specializing to new datasets. This opens new routes for creating pre-trained, foundation models for interatomic potentials.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Cross-functional transferability in foundation machine learning interatomic potentials

The rapid development of foundation potentials (FPs) in machine learning interatomic potentials demonstrates the possibility for generalizable learning of the universal potential energy surface. The accuracy of FPs can be further improved by bridging the model from lower-fidelity datasets to high-fidelity ones. In this work, we analyze the challenge of this transfer learning (TL) problem within the CHGNet framework. We show that significant energy scale shifts and poor correlations between GGA and r 2 SCAN hinder cross-functional transferability. By benchmarking different TL approaches on the MP-r 2 SCAN dataset, we demonstrate the importance of elemental energy referencing in the TL of FPs. By comparing the scaling law with and without the pre-training on a low-fidelity dataset, we show that significant data efficiency can still be achieved through TL, even with a target dataset of sub-million structures. We highlight the importance of proper TL and multi-fidelity learning in creating next-generation FPs on high-fidelity data.

Huang, Xu [University of California, Berkeley, CA ↗
DOE OSTI2021

A probabilistic graphical model foundation for enabling predictive digital twins at scale

A unifying mathematical formulation is needed to move from one-off digital twins built through custom implementations to robust digital twin implementations at scale. This work proposes a probabilistic graphical model as a formal mathematical representation of a digital twin and its associated physical asset. We create an abstraction of the asset–twin system as a set of coupled dynamical systems, evolving over time through their respective state spaces and interacting via observed data and control inputs. The formal definition of this coupled system as a probabilistic graphical model enables us to draw upon well-established theory and methods from Bayesian statistics, dynamical systems and control theory. The declarative and general nature of the proposed digital twin model make it rigorous yet flexible, enabling its application at scale in a diverse range of application areas. Here, we demonstrate how the model is instantiated to enable a structural digital twin of an unmanned aerial vehicle (UAV). The digital twin is calibrated using experimental data from a physical UAV asset. Its use in dynamic decision-making is then illustrated in a synthetic example where the UAV undergoes an in-flight damage event and the digital twin is dynamically updated using sensor data. The graphical model foundation ensures that the digital twin calibration and updating process is principled, unified and able to scale to an entire fleet of digital twins.

42 ENGINEERING↗
DOE OSTI2025

Crystal structure prediction with host-guided inpainting generation and foundation potentials

Unconditional crystal structure generation with diffusion models faces challenges in identifying symmetric crystals as the unit cell size increases. Here, we present the crystal host-guided generation (CHGGen) framework to address this challenge through conditional generation using an inpainting method, which optimizes a fraction of atomic positions within a predefined and symmetrized host structure to improve the success rate for symmetric structure generation. By integrating inpainting structure generation with a foundation potential for structure optimization, we demonstrate the method on the ZnS–P 2 S 5 and Li–Si chemical systems, where the inpainting method generates a higher fraction of symmetric structures than unconditional generation. The practical significance of CHGGen extends to enabling the structural modification of crystal structures, particularly for systems with partial occupancy or intercalation chemistry. The inpainting method also allows for seamless integration with other generative models, providing a versatile framework for accelerating materials discovery.

Zhong, Peichen [University of California, Berkeley↗
DOE OSTI2024

Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models

Abstract We propose masked particle modeling (MPM) as a self-supervised method for learning generic, transferable, and reusable representations on unordered sets of inputs for use in high energy physics (HEP) scientific data. This work provides a novel scheme to perform masked modeling based pre-training to learn permutation invariant functions on sets. More generally, this work provides a step towards building large foundation models for HEP that can be generically pre-trained with self-supervised learning and later fine-tuned for a variety of down-stream tasks. In MPM, particles in a set are masked and the training objective is to recover their identity, as defined by a discretized token representation of a pre-trained vector quantized variational autoencoder. We study the efficacy of the method in samples of high energy jets at collider physics experiments, including studies on the impact of discretization, permutation invariance, and ordering. We also study the fine-tuning capability of the model, showing that it can be adapted to tasks such as supervised and weakly supervised jet classification, and that the model can transfer efficiently with small fine-tuning data sets to new classes and new data domains.

Heinrich, Lukas (ORCID:0000000240487584)↗
DOE OSTI2024

Maven: a multimodal foundation model for supernova science

Abstract A common setting in astronomy is the availability of a small number of high-quality observations, and larger amounts of either lower-quality observations or synthetic data from simplified models. Time-domain astrophysics is a canonical example of this imbalance, with the number of supernovae observed photometrically outpacing the number observed spectroscopically by multiple orders of magnitude. At the same time, no data-driven models exist to understand these photometric and spectroscopic observables in a common context. Contrastive learning objectives, which have grown in popularity for aligning distinct data modalities in a shared embedding space, provide a potential solution to extract information from these modalities. We present Maven, the first foundation model for supernova science. To construct Maven, we first pre-train our model to align photometry and spectroscopy from 0.5 M synthetic supernovae using a contrastive objective. We then fine-tune the model on 4702 observed supernovae from the Zwicky transient facility. Maven reaches state-of-the-art performance on both classification and redshift estimation, despite the embeddings not being explicitly optimized for these tasks. Through ablation studies, we show that pre-training with synthetic data improves overall performance. In the upcoming era of the Vera C. Rubin observatory, Maven will serve as a valuable tool for leveraging large, unlabeled and multimodal time-domain datasets.

Zhang, Gemma (ORCID:0000000280198082)↗
DOE OSTI2024

CryoSegNet: accurate cryo-EM protein particle picking by integrating the foundational AI image segmentation model and attention-gated U-Net

Picking protein particles in cryo-electron microscopy (cryo-EM) micrographs is a crucial step in the cryo-EM-based structure determination. However, existing methods trained on a limited amount of cryo-EM data still cannot accurately pick protein particles from noisy cryo-EM images. The general foundational artificial intelligence–based image segmentation model such as Meta’s Segment Anything Model (SAM) cannot segment protein particles well because their training data do not include cryo-EM images. Here, we present a novel approach (CryoSegNet) of integrating an attention-gated U-shape network (U-Net) specially designed and trained for cryo-EM particle picking and the SAM. The U-Net is first trained on a large cryo-EM image dataset and then used to generate input from original cryo-EM images for SAM to make particle pickings. CryoSegNet shows both high precision and recall in segmenting protein particles from cryo-EM micrographs, irrespective of protein type, shape and size. On several independent datasets of various protein types, CryoSegNet outperforms two top machine learning particle pickers crYOLO and Topaz as well as SAM itself. The average resolution of density maps reconstructed from the particles picked by CryoSegNet is 3.33 Å, 7% better than 3.58 Å of Topaz and 14% better than 3.87 Å of crYOLO. It is publicly available at https://github.com/jianlin-cheng/CryoSegNet

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Ethical, legal and social implications of human genome studies in radiation research: a workshop report for studies on atomic bomb survivors at the Radiation Effects Research Foundation

The Radiation Effects Research Foundation (RERF) is the primary organization in Japan dedicated to studying the health consequences of the Hiroshima and Nagasaki atomic bombings in World War II. In December 2020, RERF held a virtual international workshop on the ethical, legal and social implications (ELSI) of genome studies. In this workshop, the ELSI considerations of future human genome studies on radiation research including atomic bomb survivors and their families were discussed. Since genome sequencing (GS) is now practical and affordable, RERF now plans GS of parents/child trios to examine genetic effects of atomic bomb radiation. As such studies may engender some novel risks and benefits, ethics review and engagement with families (including consent) need to be considered. These include protection of individual privacy, use of samples from deceased prior participants, return of results to the participants, public sharing of genome data and advance science and social welfare. Specifically with regard to social welfare, the results of such studies may have implications for public and government decision-making regarding social benefits of victims and other important questions. Based on these broad-ranging discussions we have developed the following concepts to guide this work: “trust,” “compromise” and “relationship building,” inclusive of the concerned stakeholders, scientific aims and Japanese society at large. We conclude that in order to realize, establish and maintain these concepts, it is essential to put procedures into place to ensure the successful, consensus-based implementation of the RERF studies.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗
DOE OSTI2023

Towards an astronomical foundation model for stars with a transformer-based model

ABSTRACT Rapid strides are currently being made in the field of artificial intelligence using transformer-based models like Large Language Models (LLMs). The potential of these methods for creating a single, large, versatile model in astronomy has not yet been explored. In this work, we propose a framework for data-driven astronomy that uses the same core techniques and architecture as used by LLMs. Using a variety of observations and labels of stars as an example, we build a transformer-based model and train it in a self-supervised manner with cross-survey data sets to perform a variety of inference tasks. In particular, we demonstrate that a single model can perform both discriminative and generative tasks even if the model was not trained or fine-tuned to do any specific task. For example, on the discriminative task of deriving stellar parameters from Gaia XP spectra, we achieve an accuracy of 47 K in Teff, 0.11 dex in log g, and 0.07 dex in [M/H], outperforming an expert XGBoost model in the same setting. But the same model can also generate XP spectra from stellar parameters, inpaint unobserved spectral regions, extract empirical stellar loci, and even determine the interstellar extinction curve. Our framework demonstrates that building and training a single foundation model without fine-tuning using data and parameters from multiple surveys to predict unmeasured observations and parameters is well within reach. Such ‘Large Astronomy Models’ trained on large quantities of observational data will play a large role in the analysis of current and future large surveys.

Leung, Henry W. (ORCID:0000000200362752)↗
DOE OSTI2025

Resimulation-based self-supervised learning for pretraining physics foundation models

Self-supervised learning (SSL) is at the core of training modern large machine learning models, providing a scheme for learning powerful representations that can be used in a variety of downstream tasks. However, SSL strategies must be adapted to the type of training data and downstream tasks required. We propose resimulation-based self-supervised representation learning (RS3L), a novel simulation-based SSL strategy that employs a method of resimulation to drive data augmentation for contrastive learning in the physical sciences, particularly, in fields that rely on stochastic simulators. By intervening in the middle of the simulation process and rerunning simulation components downstream of the intervention, we generate multiple realizations of an event, thus producing a set of augmentations covering all physics-driven variations available in the simulator. Using experiments from high-energy physics, we explore how this strategy may enable the development of a foundation model; we show how RS3L pretraining enables powerful performance in downstream tasks such as discrimination of a variety of objects and uncertainty mitigation. In addition to our results, we make the RS3L dataset publicly available for further studies on how to improve SSL strategies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Constrained GAN-Generated X-Ray CT Data For Self-Supervised And Foundation-Model Segmentation Of Concrete Microstructures

Three-dimensional characterization of materials using X-ray computed tomography (XCT) is challenging due to the complexity of internal structures, noise, and variations in resolution. Traditional computer vision models often struggle to accurately segment these images, particularly in domain-specific applications like materials science. While supervised deep learning approaches have been developed to address the limitations of conventional algorithms, they typically require large amounts of labeled training data and often fail to generalize across different datasets. Self-supervised, few-and zero-shot learning methods have gained prominence in natural image processing and segmentation tasks, but their application to scientific imaging remains limited due to the unique structural complexity, noise, and textural artifacts present in materials science data. In this work, we investigate how domain adaptation, leveraging physics-based and GAN-generated synthetic data, impacts segmentation performance. We introduce a modified Contrastive Unpaired Translation (CUT) model designed to generate realistic labeled data, which can be used for training, pre-training, and fine-tuning segmentation models for real XCT microstructure data. We evaluate the performance of two segmentation approaches: a self-supervised network (SSL-ALPNet) and a foundation model (Segment Anything Model), assessing their improvements when pre-trained and/or fine-tuned on the synthesized data. Our results demonstrate that leveraging synthetic data significantly enhances segmentation performance, particularly in challenging materials science applications.

Ziabari, Amir [ORNL] (ORCID:000000034776457X)↗
DOE OSTI2024

Pretraining Billion-Scale Geospatial Foundational Models on Frontier

As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.

Tsaris, Aristeidis (aris)↗