Search NASA⌕ Search

SEARCH · Search NASA

Results for “pruning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Towards provably efficient quantum algorithms for large-scale machine-learning models

Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, we show that fault-tolerant quantum computing could possibly provide provably efficient resolutions for generic (stochastic) gradient descent algorithms, scaling as $\mathcal{O}$(T 2 x polylog($n$)), where n is the size of the models and T is the number of iterations in the training, as long as the models are both sufficiently dissipative and sparse, with small learning rates. Based on earlier efficient quantum algorithms for dissipative differential equations, we find and prove that similar algorithms work for (stochastic) gradient descent, the primary algorithm for machine learning. In practice, we benchmark instances of large machine learning models from 7 million to 103 million parameters. We find that, in the context of sparse training, a quantum enhancement is possible at the early stage of learning after model pruning, motivating a sparse parameter download and re-upload scheme. Our work shows solidly that fault-tolerant quantum algorithms could potentially contribute to most state-of-the-art, large-scale machine-learning problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Comparing crystal structures with symmetry and geometry

Abstract Measuring the similarity between two arbitrary crystal structures is a common challenge in crystallography and materials science. Although there are an infinite number of ways to mathematically relate two crystal structures, only a few are physically meaningful. Here we introduce both a geometry-based and a symmetry-adapted similarity metric to compare crystal structures. Using crystal symmetry and combinatorial optimization we describe an algorithm to arrive at the structural relationship that minimizes these similarity metrics across all possible maps between any pair of crystal structures. The approach makes it possible to (i) identify pairs of crystal structures that are identical, (ii) quantitatively measure the similarity between crystal structures, and (iii) find and rank structural transformation pathways between any pair of crystal structures. We discuss the advantages of using the symmetry-adapted cost metric over the geometric cost. Finally, we show that all known structural transformation pathways between common crystal structures are recovered with the mapping algorithm. The methodology presented in this study will be of value to efforts that seek to catalogue crystal structures, identify structural transformation pathways or prune large first-principles datasets used to parameterize on-lattice Hamiltonians.

36 MATERIALS SCIENCE↗
DOE OSTI2014

Adaptive multiconfigurational wave functions

A method is suggested to build simple multiconfigurational wave functions specified uniquely by an energy cutoff Λ. These are constructed from a model space containing determinants with energy relative to that of the most stable determinant no greater than Λ. The resulting Λ-CI wave function is adaptive, being able to represent both single-reference and multireference electronic states. We also consider a more compact wave function parameterization (Λ+SD-CI), which is based on a small Λ-CI reference and adds a selection of all the singly and doubly excited determinants generated from it. We report two heuristic algorithms to build Λ-CI wave functions. The first is based on an approximate prescreening of the full configuration interaction space, while the second performs a breadth-first search coupled with pruning. The Λ-CI and Λ+SD-CI approaches are used to compute the dissociation curve of N{sub 2} and the potential energy curves for the first three singlet states of C{sub 2}. Special attention is paid to the issue of energy discontinuities caused by changes in the size of the Λ-CI wave function along the potential energy curve. This problem is shown to be solvable by smoothing the matrix elements of the Hamiltonian. Our last example, involving the Cu{sub 2}O{sub 2}{sup 2+} core, illustrates an alternative use of the Λ-CI method: as a tool to both estimate the multireference character of a wave function and to create a compact model space to be used in subsequent high-level multireference coupled cluster computations.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2014

Monte Carlo simulations of lattice models for single polymer systems

Single linear polymer chains in dilute solutions under good solvent conditions are studied by Monte Carlo simulations with the pruned-enriched Rosenbluth method up to the chain length N∼O(10{sup 4}). Based on the standard simple cubic lattice model (SCLM) with fixed bond length and the bond fluctuation model (BFM) with bond lengths in a range between 2 and √(10), we investigate the conformations of polymer chains described by self-avoiding walks on the simple cubic lattice, and by random walks and non-reversible random walks in the absence of excluded volume interactions. In addition to flexible chains, we also extend our study to semiflexible chains for different stiffness controlled by a bending potential. The persistence lengths of chains extracted from the orientational correlations are estimated for all cases. We show that chains based on the BFM are more flexible than those based on the SCLM for a fixed bending energy. The microscopic differences between these two lattice models are discussed and the theoretical predictions of scaling laws given in the literature are checked and verified. Our simulations clarify that a different mapping ratio between the coarse-grained models and the atomistically realistic description of polymers is required in a coarse-graining approach due to the different crossovers to the asymptotic behavior.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2015

Improving the efficiency of branch-and-bound complete-search NMR assignment using the symmetry of molecules and spectra

Nuclear magnetic resonance (NMR) assignment of small molecules is presented as a typical example of a combinatorial optimization problem in chemical physics. Three strategies that help improve the efficiency of solution search by the branch and bound method are presented: 1. reduction of the size of the solution space by resort to a condensed structure formula, wherein symmetric nuclei are grouped together; 2. partitioning of the solution space based on symmetry, that becomes the basis for an efficient branching procedure; and 3. a criterion of selection of input restrictions that leads to increased gaps between branches and thus faster pruning of non-viable solutions. Although the examples chosen to illustrate this work focus on small-molecule NMR assignment, the results are generic and might help solving other combinatorial optimization problems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2016

Assessing the utility of phase-space-localized basis functions: Exploiting direct product structure and a new basis function selection procedure

In this paper we show that it is possible to use an iterative eigensolver in conjunction with Halverson and Poirier’s symmetrized Gaussian (SG) basis [T. Halverson and B. Poirier, J. Chem. Phys. 137, 224101 (2012)] to compute accurate vibrational energy levels of molecules with as many as five atoms. This is done, without storing and manipulating large matrices, by solving a regular eigenvalue problem that makes it possible to exploit direct-product structure. These ideas are combined with a new procedure for selecting which basis functions to use. The SG basis we work with is orders of magnitude smaller than the basis made by using a classical energy criterion. We find significant convergence errors in previous calculations with SG bases. For sum-of-product Hamiltonians, SG bases large enough to compute accurate levels are orders of magnitude larger than even simple pruned bases composed of products of harmonic oscillator functions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Maximizing efficiency of dataset compression for machine learning potentials with information theory

Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Spatial Correlations of the Poisson Model for Radiation Transport

Characterizing the relationship between bulk physical properties and mixing in randomly heterogeneous media is a central challenge across many areas of science and engineering. A benchmark model for such studies is the Poisson model, a random tessellation of space by a Poisson process of hyperplanes. In radiation transport studies, the lack of exact expressions for the Poisson model’s spatial multipoint functions has led to approximate methods being used, introducing unquantified sources of error. Here, we recently introduced an exact solution for the Poisson model’s multipoint functions and closely related conditional probability functions (CPFs), providing a new opportunity to understand and reduce these sources of error. In this paper, we enable a more rigorous investigation of radiation transport in stochastic media by applying the recently introduced exact solution for the Poisson model’s CPFs. This paper consists of three main contributions. First, we introduce a unified framework for CPFs of the Poisson model, encompassing the recently introduced exact CPFs as well as the previously introduced atomic mix, nearest-neighbor, and combination CPFs. This framework also includes existing pruning techniques for the approximate CPFs, such as angular exclusion, as well as a novel form of angular exclusion suitable for the exact CPFs. Second, we use the exact CPFs to characterize the spatial regions where each approximate three-point CPF is most accurate, thereby explaining the observed hierarchy of accuracy among the approximate models. Finally, we evaluate material transmittance, reflectance, and flux in a three-dimensional test problem using conditional point sampling, demonstrating the relationship between CPF accuracy and transport simulation accuracy.

Poisson model↗
DOE OSTI2014

THE SAMI GALAXY SURVEY: TOWARD A UNIFIED DYNAMICAL SCALING RELATION FOR GALAXIES OF ALL TYPES

We take advantage of the first data from the Sydney-AAO Multi-object Integral field Galaxy Survey to investigate the relation between the kinematics of gas and stars, and stellar mass in a comprehensive sample of nearby galaxies. We find that all 235 objects in our sample, regardless of their morphology, lie on a tight relation linking stellar mass (M {sub *}) to internal velocity quantified by the S {sub 0.5} parameter, which combines the contribution of both dispersion (σ) and rotational velocity (V {sub rot}) to the dynamical support of a galaxy (S{sub 0.5}=√(0.5 V{sub rot}{sup 2}+σ{sup 2})). Our results are independent of the baryonic component from which σ and V {sub rot} are estimated, as the S {sub 0.5} of stars and gas agree remarkably well. This represents a significant improvement compared to the canonical M {sub *} versus V {sub rot} and M {sub *} versus σ relations. Not only is no sample pruning necessary, but also stellar and gas kinematics can be used simultaneously, as the effect of asymmetric drift is taken into account once V {sub rot} and σ are combined. Our findings illustrate how the combination of dispersion and rotational velocities for both gas and stars can provide us with a single dynamical scaling relation valid for galaxies of all morphologies across at least the stellar mass range 8.5 <log (M {sub *}/M {sub ☉}) < 11. Such relation appears to be more general and at least as tight as any other dynamical scaling relation, representing a unique tool for investigating the link between galaxy kinematics and baryonic content, and a less biased comparison with theoretical models.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Fast convolutional neural networks on FPGAs with hls4ml

We introduce an automated tool for deploying ultra low-latency, low-power deep neural networks with convolutional layers on field-programmable gate arrays (FPGAs). By extending the hls4ml library, we demonstrate an inference latency of 5 µs using convolutional architectures, targeting microsecond latency applications like those at the CERN Large Hadron Collider. Considering benchmark models trained on the Street View House Numbers Dataset, we demonstrate various methods for model compression in order to fit the computational constraints of a typical FPGA device used in trigger and data acquisition systems of particle detectors. In particular, we discuss pruning and quantization-aware training, and demonstrate how resource utilization can be significantly reduced with little to no loss in model accuracy. We show that the FPGA critical resource consumption can be reduced by 97% with zero loss in model accuracy, and by 99% when tolerating a 6% accuracy degradation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Distilling particle knowledge for fast reconstruction at high-energy physics experiments

Knowledge distillation is a form of model compression that allows artificial neural networks of different sizes to learn from one another. Its main application is the compactification of large deep neural networks to free up computational resources, in particular on edge devices. In this article, we consider proton-proton collisions at the High-Luminosity Large Hadron Collider (HL-LHC) and demonstrate a successful knowledge transfer from an event-level graph neural network (GNN) to a particle-level small deep neural network (DNN). Our algorithm, DistillNet, is a DNN that is trained to learn about the provenance of particles, as provided by the soft labels that are the GNN outputs, to predict whether or not a particle originates from the primary interaction vertex. The results indicate that for this problem, which is one of the main challenges at the HL-LHC, there is minimal loss during the transfer of knowledge to the small student network, while improving significantly the computational resource needs compared to the teacher. This is demonstrated for the distilled student network on a CPU, as well as for a quantized and pruned student network deployed on a field programmable gate array. Our study proves that knowledge transfer between networks of different complexity can be used for fast artificial intelligence (AI) in high-energy physics that improves the expressiveness of observables over non-AI-based reconstruction algorithms. Such an approach can become essential at the HL-LHC experiments, e.g. to comply with the resource budget of their trigger stages.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

Algorithm sensitivity analysis and parameter tuning for tissue image segmentation pipelines

Motivation: Sensitivity analysis and parameter tuning are important processes in large-scale image analysis. They are very costly because the image analysis workflows are required to be executed several times to systematically correlate output variations with parameter changes or to tune parameters. An integrated solution with minimum user interaction that uses effective methodologies and high performance computing is required to scale these studies to large imaging datasets and expensive analysis workflows. Results: The experiments with two segmentation workflows show that the proposed approach can (i) quickly identify and prune parameters that are non-influential; (ii) search a small fraction (about 100 points) of the parameter search space with billions to trillions of points and improve the quality of segmentation results (Dice and Jaccard metrics) by as much as 1.42 x compared to the results from the default parameters; (iii) attain good scalability on a high performance cluster with several effective optimizations. Conclusions: Our work demonstrates the feasibility of performing sensitivity analyses, parameter studies and auto-tuning with large datasets. The proposed framework can enable the quantification of error estimations and output variations in image segmentation pipelines.

Biochemistry & Molecular Biology↗
DOE OSTI2022

An agent-based blackboard system for multi-objective optimization

In the field of multi-objective optimization, there are a multitude of algorithms from which to choose. Each algorithm has strengths and weaknesses associated with the mechanics for finding the Pareto front. Recently, researchers have begun to examine how multi-agent environments can be used to help solve multi-objective optimization problems. In this work, we propose a multi-objective optimization algorithm based on a multi-agent blackboard system (MABS). The MABS framework allows for multiple agents to read and write pertinent optimization problem data to a central blackboard agent. Agents can stochastically search the design space, use previously discovered solutions to explore local optima, or update and prune the Pareto front. A centralized blackboard framework allows the optimization problem to be solved in a cohesive manner and permits stopping, restarting, or updating the optimization problem. The MABS framework is tested against three alternative optimization algorithms across a suite of engineering design problems and typically outperforms the other algorithms in discovering the Pareto front. A parallelizability study is performed where we find that the MABS is able to evaluate a set number of designs, which require an evaluation time ranging from 0 to 300 seconds, quicker than a traditional optimization algorithm: this fact becomes more apparent the longer it takes to evaluate a design. To provide context for the benefits provided by MABS, a real-world nuclear engineering design problem is examined. MABS is used to examine the placement of experiments in a nuclear reactor, where we are able to evaluate hundreds of configurations for experimental placement while maintaining a strict set of safety constraints.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

The impact of human expert visual inspection on the discovery of strong gravitational lenses

We investigate the ability of human ‘expert’ classifiers to identify strong gravitational lens candidates in Dark Energy Survey like imaging. We recruited a total of 55 people that completed more than 25 percent of the project. During the classification task, we present to the participants 1489 images. The sample contains a variety of data including lens simulations, real lenses, non-lens examples, and unlabelled data. We find that experts are extremely good at finding bright, well-resolved Einstein rings, while arcs with g-band signal to noise less than ~25 or Einstein radii less than ~1.2 times the seeing are rarely recovered. Very few non-lenses are scored highly. There is substantial variation in the performance of individual classifiers, but they do not appear to depend on the classifier’s experience, confidence or academic position. These variations can be mitigated with a team of 6 or more independent classifiers. Our results give confidence that humans are a reliable pruning step for lens candidates, providing pure and quantifiably complete samples for follow-up studies.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

The crystal structure of DR6 in complex with the amyloid precursor protein provides insight into death receptor activation

The amyloid precursor protein (APP) has garnered considerable attention due to its genetic links to Alzheimer's disease. Death receptor 6 (DR6) was recently shown to bind APP via the protein extracellular regions, stimulate axonal pruning, and inhibit synapse formation. Here, we report the crystal structure of the DR6 ectodomain in complex with the E2 domain of APP and show that it supports a model for APP-induced dimerization and activation of cell surface DR6.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2018

On the application of the expected log-likelihood gain to decision making in molecular replacement

Molecular-replacement phasing of macromolecular crystal structures is often fast, but if a molecular-replacement solution is not immediately obtained the crystallographer must judge whether to pursue molecular replacement or to attempt experimental phasing as the quickest path to structure solution. The introduction of the expected log-likelihood gain has given the crystallographer a powerful new tool to aid in making this decision. The eLLG is the log-likelihood gain on intensity expected from a correctly placed model. It is calculated as a sum over the reflections of a function dependent on the fraction of the scattering for which the model accounts, the estimated model coordinate error and the measurement errors in the data. It is shown how the eLLG may be used to answer the question 'can I solve my structure by molecular replacement?'. However, this is only the most obvious of the applications of the eLLG. It is also discussed how the eLLG may be used to determine the search order and minimal data requirements for obtaining a molecular-replacement solution using a given model, and for decision making in fragment-based molecular replacement, single-atom molecular replacement and likelihood-guided model pruning.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Clustering-Based Predictive Analytics to Improve Scientific Data Discovery

Given the sheer volume of scientific data archived within the data-intensive projects at the US Department of Energy's Oak Ridge National Laboratory, finding precisely what data we are looking for may not be a trivial task; conversely, we may also miss a more prominent data product. To address such issues, we propose improving the data discovery system and using data analytics methods to comprehend what specific users might be interested in based on their physiological state, search patterns, and past data usage history. This work's primary goal is to prune the complexity, increase the visibility of popular data products, and direct users toward the data that best meet their needs. The proposed algorithm constructs a user profile based on the user's explicit or implicit interactions with the system, such as items they are currently looking at on-site and the key metadata mappings related to the data set. The pattern is then used to build a training data set, which will help find relevant data to recommend to the user.

Devarakonda, Ranjeet↗
DOE OSTI2025

The Effects of Compounded Model Size Reductions on Adversarial Robustness

Recent advances in Edge AI and Tiny Machine Learning (TinyML) have enabled the deployment of machine learning models on resource-constrained environments. However, deploying these models on edge devices, such as micro-controllers, requires significant model footprint reduction through a variety of techniques such as quantization, pruning, and clustering. While these optimization methods offer considerable advantages, they potentially introduce AI-related security vulnerabilities, particularly concerning model robustness with respect to adversarial AI attacks. Prior research has extensively examined the impact of quantization on adversarial robustness; however, the effects of alternative reduction techniques and their combinations remain understudied. This paper investigates the impact of model size reduction techniques on adversarial robustness, when applied individually and combined. We utilized Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks to generate adversarial perturbations for both training and testing data, and then evaluated the models' accuracy under adversarial training conditions. Our findings revealed that reduction techniques generally diminished robustness; although, combining techniques was not found to make robustness any worse than when applied individually. Moreover, specific techniques can potentially enhance resistance to small size perturbations. This research provides insights into the trade-offs between model size reduction and security, establishing a foundation for future investigations into improving adversarial training techniques and methodologies for maintaining robustness while preserving memory footprint benefits.

Austria, Phillipe [ORNL] (ORCID:0000000236223973)↗