Search NASA⌕ Search

SEARCH · Search NASA

Results for “small scale processes”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Recent Advances of PyROS: A Pyomo Solver for Nonconvex Two-Stage Robust Optimization in Process Systems Engineering

This poster highlights uncertainty and technical risk reduction capabilities in CCSI2, with a focus on robust optimization. It presents recent advances of the two-stage robust optimization (RO) solver PyROS and applications to advanced energy systems optimization. To demonstrate the computational performance and reliability of PyROS, a benchmarking study on a library of over 8,500 small-scale RO problems is presented. Further, PyROS is used to obtain robust system designs of a MEA-based CO2 absorber under uncertainty in the thermodynamic property models for a variety of CO2 capture rate threshold requirements. Overall, the results demonstrate that the PyROS solver, including recent extensions to multi-stage RO settings, provides a reliable avenue to optimize the design and operation of advanced energy systems subject to various sources of parametric uncertainty.

Sherman, Jason↗

Revisiting single inclusive jet production: timelike factorization and reciprocity

Factorization theorems for single inclusive jet production play a crucial role in the study of jets and their substructure. In the case of small radius jets, the dynamics of the jet clustering can be factorized from both the hard production dynamics, and the dynamics of the low scale jet substructure measurement, and is described by a matching coefficient that can be computed in perturbative Quantum Chromodynamics (QCD). A proposed factorization formula describing this process has been previously presented in the literature, and is referred to as the semi-inclusive, or fragmenting jets formalism. By performing an explicit two-loop calculation, we show the inconsistency of this factorization formula, in agreement with another recent result in the literature. Building on recent progress in the factorization of single logarithmic observables, and the understanding of reciprocity, we then derive a new all-order factorization theorem for inclusive jet production. The use of a jet algorithm, being only a modification of the infrared structure of the measurement, modifies the structure of convolutions in the factorization theorem, as compared to inclusive fragmentation, but maintains the universality of the inclusive hard function and its associated Dokshitzer-Gribov-Lipatov-Altarelli-Parisi (DGLAP) evolution, which are ultraviolet properties. However, the non-trivial structure of convolutions in the factorization theorem implies that the jet functions exhibit a modified evolution. We perform an explicit two-loop calculation of the jet function in both N = 4 super Yang-Mills (SYM), and for all color channels in QCD, finding exact agreement with the structure derived from our renormalization group equations. In addition, we derive several new results, including an extension of our factorization formula to jet substructure observables, a jet algorithm definition of a generating function for the energy correlators, and new results for exclusive jet functions. Our results are a key ingredient for achieving precision jet substructure at colliders.

Effective Field Theories↗

Constraining Galaxy-Halo connection using machine learning

We investigate the potential of machine learning (ML) methods to model small-scale galaxy clustering for constraining Halo Occupation Distribution (HOD) parameters. Our analysis reveals that while many ML algorithms report good statistical fits, they often yield likelihood contours that are significantly biased in both mean values and variances relative to the true model parameters. This highlights the importance of careful data processing and algorithm selection in ML applications for galaxy clustering, as even seemingly robust methods can lead to biased results if not applied correctly. ML tools offer a promising approach to exploring the HOD parameter space with significantly reduced computational costs compared to traditional brute-force methods if their robustness is established. Using our ANN-based pipeline, we successfully recreate some standard results from recent literature. Properly restricting the HOD parameter space, transforming the training data, and carefully selecting ML algorithms are essential for achieving unbiased and robust predictions. Among the methods tested, artificial neural networks (ANNs) outperform random forests (RF) and ridge regression in predicting clustering statistics, when the HOD prior space is appropriately restricted. We demonstrate these findings using the projected two-point correlation function (w p (r p )), angular multipoles of the correlation function (ξ ℓ (r)), and the void probability function (VPF) of Luminous Red Galaxies from Dark Energy Spectroscopic Instrument mocks. Our results show that while combining w p (r p ) and VPF improves parameter constraints, adding the multipoles ξ 0 , ξ 2 , and ξ 4 to w p (r p ) does not significantly improve the constraints.

cosmology↗

Understanding and Controlling Photoexcited Molecules in Complex Environments (Final Technical Report)

Predicting the outcome of a photochemical reaction is complicated by factors outside the scope of tradition theoretical chemistry. On such small scales, fluctuations of the environment render outcomes stochastic, nonadiabatic dynamics blur the separation between nuclear and electronic motion, and ultrafast relaxation renders typical assumptions of equilibrium inappropriate. This project aimed to elucidate and control nonadiabatic molecular dynamics in condensed phases. We developed accurate, efficient simulation tools to study irreversible molecular processes computational across a range of scales. We applied these methodological advances to study electron transfer under exotic conditions where spin and topology opened new pathways for reactivity. We also brought these ideas to bare on light harvesting systems, where energy and charge transport are coupled, and where long lifetimes can be deleterious to function. These advances helped to establish general principles for chemical efficiency and guide the design of nanoscale energy materials, molecular machines, and related technologies.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Data Summarization and Inference at Scale

This is the final report for the DOE ASCR grant SC-0022260, Data Summarization and Inference at Scale, PI: Alex Pothen, Purdue University. The goal of the project was to solve data-intensive and compute-intensive problems in the physical sciences, engineering, information science, data science, etc. by designing and implementing new algorithms that could work with a subset of the data. The four subgoals were: (a) The solution of problems where the data is too large to be stored in the memory of a computer. In this streaming model of computation, the data arrives as a stream of elements to the computer, each element is processed as it arrives, and a decision is made to discard the data or to store it; only a small subset of the data proportional to the size of the output solution is stored, and when all the data has been streamed, a solution to the problem is computed from the stored subset. (b) The use of machine learning methods to compute solutions to data-intensive problems. The use of GPUs is critical to obtain high performance on machine learning tasks, but their memory sizes are smaller relative to that of CPUs. For large-scale problems, the data is sampled many times, and small samples are used with repetition, for robustness, to compute solutions to inference tasks. This sampling reduces the memory required to solve the problem, but attention is needed to avoid slow convergence to the solutions, and reduced accuracy of inference. We propose submodular optimization, Large Language Models, and physics-informed neural networks to enable GPU computations here. (c) Modeling and visualization of high-dimensional data using interpretable features. Clinical proteomic data sets from immunology for the detection of cancer and other diseases are temporal and high-dimensional, and algorithms for visualizing these data sets using clinically interpretable features are lacking. We propose methods that compute distances based on the optimal transportation problem and graph edit distances to address this problem. We also propose the use of optimal transport-based distances, spatial statistics, and network structure to classify image data sets, We apply these algorithms to electron micrographs of the peripheral nervous system in the digestive tract. (d) The design of data-intensive algorithms on emerging architectures, specifically, noisy, intermediate-scale quantum (NISQ) devices. Quantum computers offer the possibility of exploring large solution spaces due to the principle of superposition, but current quantum computers are limited by few qubits, short coherence times due to noise, poor interconections among the qubits, etc. We propose the use of the divide and conquer paradigm to solve large-scale problems, wherein collections of small subproblems are solved on the quantum devices, and the solutions to the subproblems are integrated into a solution for the original problem on a classical computer.

97 MATHEMATICS AND COMPUTING↗

Jacobian-based model diagnostics and application to equation oriented modeling of a carbon capture system

It can be difficult to identify the specific variables or equations responsible for convergence issues in large mathematical programming models. The Institute for the Design of Advanced Energy Systems Integrated Platform (IDAES-IP) contains a tool to identify poorly scaled constraints and variables by searching for rows and columns of the Jacobian matrix with small L2-norms. A singular value decomposition is then performed to identify degenerate sets of equations and remaining scaling issues. Here, this work presents a flowsheet developed for post-combustion carbon capture using a monoethanolamine (MEA) solvent system as a case study. This work takes the reader through the entire process of model diagnostics and reformulation, from a basic introduction to the mathematics behind these model diagnostics to the reformulations necessary to make the model numerically robust, including a significantly modified enhancement factor model.

IDAES↗

From molecular to macroscopic: predicting liquid–liquid phase equilibria and small-angle scattering of mixtures of organic liquids from atomistic simulation using Kirkwood–Buff theory

Macroscopic phase equilibria between solutions define the functionality of many biological and industrial processes, yet they are challenging to predict due to the inherent complexity of liquids containing large molecules. This work introduces an approach for the purely predictive calculation of such phase equilibria in temperature-composition space from molecular dynamics (MD) simulations at one temperature in the single-phase region. We use an approach developed previously to obtain the entropic and enthalpic contributions to the free energy of mixing from the atomic-scale information given by MD simulations via Kirkwood–Buff theory. This allows us to accurately estimate the free energy of mixing as a function of temperature, and thus obtain liquid–liquid phase equilibria, including liquid–liquid critical points, associated binodal and spinodal lines, and composition fluctuations across a region of temperature and composition. Results for binary malonamide–alkane systems are validated by comparison to a direct experimental probe of the fluctuations: the small angle X-ray scattering intensity near zero wavenumber. The MDKB → Phase method demonstrated here provides a significant improvement in predicting liquid–liquid equilibria and free energy as a function of temperature for our systems of interest compared to conventional thermodynamic models. The accurate performance of this purely predictive approach lies in its preservation of atomistic details when determining thermodynamic properties. Furthermore, its inherent extensibility to multi-component systems will likely make the MDKB → Phase approach a valuable general tool for connecting molecular interactions to macroscopic phase equilibria and for the computational screening of materials for targeted thermodynamic behavior.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Robust surfactant-assisted one-pot sample preparation for label-free single-cell and nanoscale proteomics

With advanced mass spectrometry (MS)-based proteomics, genome-scale proteome coverage can be achieved from bulk cells. However, such bulk measurement obscures cell to cell heterogeneity, precluding proteome profiling of single cells and small numbers of cells of interest. To address this issue, in recent 5 years there are a surge of small sample preparation methods developed for robust effective collection and processing of single cells and small numbers of cells for in-depth MS-based proteome profiling. Based on their broad accessibility, they can be categorized into two types: specific device- and standard PCR tube- or multi-well plate-based methods. Herein we describe the detailed protocol of our recently developed, easily adoptable, Surfactant-assisted One-Pot (SOP) sample preparation coupled with MS method termed SOP-MS for label-free single-cell and nanoscale proteomics. SOP-MS capitalizes on the combination of a MS-compatible surfactant, DDM (n-Dodecyl-ß-D-maltoside), and standard low-bind PCR tube or multi-well plate for ‘all-in-one’ one-pot sample preparation without sample transfer. With its robust and convenient features, SOP-MS can be readily implemented in any MS laboratory for single-cell and nanoscale proteomics. With further improvements in MS detection sensitivity and sample throughput, we believe that SOP-MS could open an avenue for single-cell proteomics with broad applicability in the biological and biomedical research.

Single-cell proteomics, nanoscale proteomics, SOP-↗

Supramolecular Assembly of Lanthanide-Binding Tag Peptides for Aqueous Separation of Rare Earth Elements

Selective and eco-friendly separation and purification methods for rare earth elements (REEs) are necessary to meet the increasing demand for these valuable metals, which are extensively used in modern electronics and clean energy technologies. Mining feedstocks consist of REE mixtures as stable trivalent cations (Ln 3+ ) that are difficult to separate due to their identical charge and similar size. Lanthanide-binding tags (LBTs), peptide chelates that coordinate Ln 3+ in binding pockets, show promise as selective, high-affinity extractants. We demonstrate that the LBT variant LBTLLA 5– , designed for high selectivity for Tb 3+ , is an effective extractant, forming complexes with REEs in solution that subsequently organize into self-assembling structures rich in Ln 3+ . These structures condense into aggregates that can be separated, enabling an efficient, all-aqueous, eco-friendly separation process. The self-assembled structures are studied using dynamic light scattering, ζ-potential measurements, transmission electron microscopy, anomalous small-angle X-ray scattering, inductively coupled plasma optical emission spectroscopy, and ultraviolet–visible absorption spectroscopy, which confirm LBTLLA 5– peptide-REE ion binding and the further assembly of micron-scale structures rich in REEs. Molecular dynamics simulations reveal the interactions promoting aggregation as well as the integrity of the binding pocket upon self-assembly. We find that LBTLLA 5– :Ln 3+ complexes recruit excess cations within the macrostructures, and we demonstrate that aggregation and selective separation can be controlled by manipulating the metal-peptide ratio in solution. Furthermore, we demonstrate separation from equimolar mixtures of REE pairs Tb 3+ -Lu 3+ and Tb 3+ -La 3+ , supporting the application of LBT peptides as a platform for the selective separation of REEs.

LBT peptides↗

ATLAS HL-LHC Demonstrators with Data Carousel: Dataon-Demand and Tape Smart Writing

The High Luminosity upgrade to the LHC (HL-LHC) is expected to deliver scientific data at the multi-exabyte scale. To tackle this unprecedented data storage challenge, the ATLAS experiment initiated the Data Carousel project in 2018. Data Carousel is a tape-driven workflow in which bulk production campaigns with input data resident on tape are executed by staging and promptly processing a sliding window to disk buffer such that only a small fraction of inputs are pinned on disk at any one time. Put in ATLAS production before Run3, Data Carousel continues to be our focus for seeking new opportunities in disk space savings, and enhancing tape usage throughout the ATLAS Distributed Computing (ADC) environment. These efforts are highlighted by two recent ATLAS HL-LHC demonstrator projects: data-on-demand and tape smart writing. In this paper, we will discuss the recent studies and outcomes from these projects. The research was conducted together with site experts at CERN and Tier-1 centers.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

CalWave's xWave Design for PacWave (Final Technical Report)

CalWave Inc. (CalWave) is developing a wave energy converter (WEC) technology that can generate electricity from ocean waves. CalWave’s design offers a unique approach to wave energy conversion that operates fully submerged and can actively adjust the wave excitation. This capability gives the architecture enhanced survivability in ocean storms without adding significant costs. Prior to this project, CalWave had completed a demonstration of a fully functional WEC system in an open ocean demonstration at nominal 1:5 scale under FOA 1663. The goal of this project was the detailed design, following relevant standards and industry best-practices, of a variant of the xWave WEC technology that can safely and efficiently operate at the DOE’s PacWave South test site for a targeted deployment of up two years. The WEC design and associated review processes proceeded in two distinct project phases: a ‘Preliminary’ and a ‘Final’ design phase. The first phase of the project consisted of the systematic design of the WEC’s key features with regards to appropriate IEC standards. The work resulted in a preliminary design of the xWave hull including structural and Power Take-Off (PTO) load estimates, as well as performance estimates for all ocean conditions the WEC would operate in at PacWave South. Following the first open-water demonstration of CalWave’s small-scale “x1” device under FOA 1663, lessons learned were fed directly into a comprehensive review of the xWave design in the second design phase of this FOA project. CalWave’s work was supported by Sandia National Lab (SNL) and the National Renewable Energy Lab (NREL) on the holistic WEC design, and detailed feedback from specialized partners on hull design, mooring and anchoring specification, and electrical grid interconnection. Optimization of the WEC system was performed using a novel numerical optimization tool developed by Sandia and optimization trends were confirmed via an experimental model scale tank test campaign. Performance estimates for PacWave and a detailed xWave design including integration of all relevant system components were concluded. The mooring design was also concluded in the Final design phase using the most up to date sea floor characterization (CPT) data.

16 TIDAL AND WAVE POWER↗

Updates to the ATLAS Data Carousel Project

The High Luminosity upgrade to the LHC (HL-LHC) is expected to deliver scientific data at the multi-exabyte scale. In order to address this unprecedented data storage challenge, the ATLAS experiment launched the Data Carousel project in 2018. Data Carousel is a tape-driven workflow whereby bulk production campaigns with input data resident on tape are executed by staging and promptly processing a sliding window to disk buffer such that only a small fraction of inputs are pinned on disk at any one time. Data Carousel is now in production for ATLAS in Run3. In this paper, we provide updates on recent Data Carousel R&D projects, including data-on-demand and tape smart writing. Data-on-demand removes from disk data that has not been accessed for a predefined period, when users request them, they will be either staged from tape or recreated by following the original production steps. Tape smart writing employs intelligent algorithms for file placement on tape in order to retrieve data back more efficiently, which is our long term strategy to achieve optimal tape usage in Data Carousel.

42 ENGINEERING↗

A GPU Accelerated Mixed‐Precision Finite Difference Informed Random Walker (FDiRW) Solver for Strongly Inhomogeneous Diffusion Problems

In nature, many complex multi‐physics coupling problems exhibit significant diffusivity inhomogeneity, where one process occurs several orders of magnitude faster than others temporally. Simulating rapid diffusion alongside slower processes demands intensive computational resources due to the necessity for small time steps. To address these computational challenges, we have developed an efficient numerical solver named Finite Difference informed Random Walker (FDiRW). In this study, we propose a GPU‐accelerated, mixed‐precision configuration for the FDiRW solver to maximize efficiency through GPU multi‐threaded parallel computation and lower precision computation. Numerical evaluation results reveal that the proposed GPU‐accelerated mixed‐precision FDiRW solver can achieve a 117× speedup over the CPU baseline, while an additional 1.75× speedup is achieved by employing lower precision GPU computation. Notably, for large model sizes, the GPU‐accelerated mixed‐precision FDiRW solver demonstrates strong scaling with the number of nodes used in simulation. When simulating radionuclide absorption processes by porous wasteform particles with a medium‐sized model of 192 × 192 × 192, this approach reduces the total computational time to 10 min, enabling the simulation of larger systems with strongly inhomogeneous diffusivity.

97 MATHEMATICS AND COMPUTING↗

EMSL Community Science Campaign Meeting: Critical Minerals and Materials - Rhizo Critical Campaign Breakout Session Report Summary

The “Critical Minerals Biogeochemistry in the Rhizosphere – Ultramafic Soils (Rhizo Critical)” campaign breakout (BO) session was organized to identify major knowledge gaps and fundamental research needs in rhizosphere microbiology and geochemistry that, if addressed, could transform our ability to recover critical minerals from ultramafic soil systems. We sought to identify significant challenges that must be surmounted in the pursuit of deeper science knowledge. Our ultimate goal is to understand this landscape well enough to identify and prioritize opportunities for EMSL to make the greatest impact with Environmental Transformations and Interactions (ETI) science area research campaigns focused on the biogeochemical processes controlling the behavior of critical minerals and materials in the rhizosphere. The increasing demand for critical materials and minerals (CMM) in the U.S. has heightened interest in low-grade ores with much attention on ultramafic soils, which contain valuable metals such as nickel (Ni), chromium (Cr), manganese, cobalt (Co), and copper (Lee et al., 2025; DOE CMM Report, 2023) used in advanced battery, magnet, wiring and wind turbines, and stainless steel technologies. Metal hyperaccumulating plants grown in ultramafic soils can extract economically valuable concentrations of CMMs through the process of phytomining. This technology has evolved from phytoremediation, which involves using plants to cleanse contaminated environments by removing, detoxifying, or stabilizing pollutants like metals and organic compounds. Hyperaccumulator plants are capable of storing metals in their living tissues at concentrations hundreds to thousands of times higher than those found in 'normal' plants. For instance, while the average concentration of Ni in the dry matter of plants growing in typical soils is usually less than 5 µg g?¹, Ni hyperaccumulation is defined by concentrations exceeding 1,000 µg g?¹ (Corzo Remigio et al., 2020; Reeves et al., 2018). Phytomining research has primarily focused on Ni (Rylott and van der Ent, 2025), for which the U.S. has very limited conventional mines in operation. Most soils typically contain Ni concentrations ranging from 7 to 50 mg kg-1, whereas serpentine soils exhibit significantly higher levels, with Ni content often ranging between 700 and 8,000 mg kg-1 (Sobczyk et al., 2017). While more than 500 plant species in over 50 different families have been identified as Ni hyperaccumulators (Kidd et al., 2018), Ni phytomining (and phytominng in general) remains largely untested because most studies are short-term, small-scale, and conducted under simplified or artificially enriched conditions, so they fail to capture the low metal concentrations, environmental variability, and management constraints that would be needed for a field-scale demonstration. Few hyperaccumulator species have been validated as true “metal crops,” and their biomass production, stress tolerance, and rooting characteristics are usually too poor to yield economically meaningful metal outputs. Critically, the basic mechanisms of metal uptake, transport, and sequestration, especially as shaped by belowground processes such as root exudation, rhizosphere chemistry, and root–microbe interactions that control metal mobility and bioavailability (Montreemuk et al., 2023; Kidd et al., 2018; Durand et al., 2023; Alford et al., 2010), are still only partially understood, and downstream metal recovery from biomass is rarely optimized. Because these limitations stem from gaps in fundamental knowledge rather than from a failure of the concept itself (Rylott and van der Ent, 2025; van der Ent et al., 2015), there is a strong need for basic science that dissects plant metal homeostasis, rhizosphere and microbial processes, and their integration with soil chemistry and process engineering to design more robust, scalable phytomining systems.

Ahkami, Amirhossein↗

Exploiting 20 Ne Isotopes for Precision Characterizations of Collectivity in Small Systems

Whether or not femto-scale droplets of quark-gluon plasma (QGP) are formed in so-called small systems at high-energy colliders is a pressing question in the phenomenology of the strong interaction. For proton-proton or proton-nucleus collisions the answer is inconclusive due to the large theoretical uncertainties plaguing the description of these processes. While upcoming data on collisions of 16 O nuclei may mitigate these uncertainties in the near future, here we demonstrate the unique possibilities offered by complementing 16 O + 16 O data with collisions of 20 Ne ions. We couple both nuclear lattice effective field theory (NLEFT) and projected generator coordinate method (PGCM) ab initio descriptions of the structure of 20 Ne and 16 O to hydrodynamic simulations of 16 O + 16 O and 20 Ne + 20 Ne collisions at high energy. We isolate the imprints of the bowling-pin shape of 20 Ne on the collective flow of hadrons, which can be used to perform quantitative tests of the hydrodynamic QGP paradigm. In particular, we predict that the elliptic flow of 20 Ne + 20 Ne collisions is enhanced by as much as 1.174⁢(8) stat ⁢(31) syst for NLEFT and 1.139⁢(6) stat ⁢(39) syst for PGCM relative to 16 O + 16 O collisions for the 1% most central events. At the same time, theoretical uncertainties largely cancel when studying relative variations of observables between two systems. This demonstrates a method based on experiments with two light-ion species for precision characterizations of the collective dynamics and its emergence in a small system.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Temporal Properties of Compressible Magnetohydrodynamic Turbulence

Describing the temporal properties of compressible magnetohydrodynamic (MHD) turbulence is a fundamental problem that has important implications for particle acceleration and transport in astrophysical plasmas. Here, by carefully analyzing the spatial and temporal properties of compressible MHD turbulence, we derive a new spectral power density function that is supported by simulations. This new function reveals that the low-frequency fluctuations are dominated by modes with small parallel wavenumbers with respect to the mean background magnetic field. Furthermore, for fluctuations with dynamically significant parallel wavenumbers, broadening around their eigenfrequencies is described by this function, which is in close agreement with simulations. We use this formalism to present the scaling properties of individual MHD modes. Such broadening is a direct consequence of nonlinear processes and is different for the three fundamental MHD modes. Our results provide a new window to investigate the temporal properties of turbulence and will enable further studies on the interaction between compressible MHD turbulence and energetic plasmas.

79 ASTRONOMY AND ASTROPHYSICS↗

Early Career: First-Principles Tools for Nonadiabatic Attosecond Dynamics in Materials

The overarching goal of this project was to develop computer tools for predicting how electrons move in molecules and solids at the attosecond (billionth-of-a-billionth of a second) time scale, during and after interaction with intense and/or high energy laser light. An associated goal was to also determine how X-ray spectroscopy could be used as a probe of these dynamics. The project resulted in multiple methodology developments that allow for these processes to be simulated from first-principles, most notably the use of small bulk-mimicking clusters to model solids and fixes for a deficiency in a commonly used method (time-dependent density functional theory). Additionally, the simulations showed that X-ray absorption peaks can be directly related to the electron density above the absorbing atom, and can thus be used as an intuitive probe of "where the electrons are" at a given time in the system. Collectively, these tools and results expected to be valuable for predicting and interpreting future attosecond experiments, especially for X-ray pump/probe studies at free-electron laser facilities.

74 ATOMIC AND MOLECULAR PHYSICS↗

GeoLoRA: Geometric integration for parameter efficient fine-tuning

Low-Rank Adaptation (LoRA) has become a widely used method for parameter-efficient fine-tuning of large-scale, pre-trained neural networks. However, LoRA and its extensions face several challenges, including the need for rank adaptivity, robustness, and computational efficiency during the fine-tuning process. We introduce GeoLoRA, a novel approach that addresses these limitations by leveraging dynamical low-rank approximation theory. GeoLoRA requires only a single backpropagation pass over the small-rank adapters, significantly reducing computational cost as compared to similar dynamical low-rank training methods and making it faster than popular baselines such as AdaLoRA. This allows GeoLoRA to efficiently adapt the allocated parameter budget across the model, achieving smaller low-rank adapters compared to heuristic methods like AdaLoRA and LoRA, while maintaining critical convergence, descent, and error-bound theoretical guarantees. The resulting method is not only more efficient but also more robust to varying hyperparameter settings. We demonstrate the effectiveness of GeoLoRA on several state-of-the-art benchmarks, showing that it outperforms existing methods in both accuracy and computational efficiency.

Schotthoefer, Steffen [ORNL] (ORCID:00000002156965↗