Search NASA⌕ Search

SEARCH · Search NASA

Results for “pruning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

“Understanding Robustness Lottery”: A Geometric Visual Comparative Analysis of Neural Network Pruning Approaches

Deep learning approaches have provided state-of-the-art performance in many applications by relying on large and overparameterized neural networks. However, such networks are very brittle and are difficult to deploy on resource-limited platforms. Model pruning, i.e., reducing the size of the network, is a widely adopted strategy that can lead to a more robust and compact model. Many heuristics exist for model pruning, but our understanding of the pruning process remains limited due to the black-box nature of a neural network model. Empirical studies show that some heuristics improve performance whereas others can make models more brittle. Here, this work aims to shed light on how different pruning methods alter the network’s internal feature representation and the corresponding impact on model performance. To facilitate a comprehensive comparison and characterization of the high-dimensional model feature space, we introduce a visual geometric analysis of feature representations. We evaluated a set of critical geometric concepts decomposed from the commonly adopted classification loss and used them to design a visualization system to compare and highlight the impact of pruning on model performance and feature representation. The proposed tool provides an environment for an in-depth comparison of pruning methods and a comprehensive understanding of how the model responds to common data corruption. By leveraging the proposed visualization, machine learning researchers can reveal the similarities between pruning methods and redundancy in robustness evaluation benchmarks, obtain geometric insights about the differences between pruned models that achieve superior robustness performance, and identify samples that are robust or fragile to model pruning and common data corruption.

Li, Zhimin [Univ. of Utah, Salt Lake City, UT (Uni↗
NASA NTRS2011

Should Pruning be a Pre-Processor of any Linear System?

There are many real-world problems whose mathematical models turn out to be linear systems Ax = b , where A is an m by x n matrix. Each equation of the linear system is an information. An information, in a physical problem, such as 4 mangoes, 6 bananas, and 5 oranges cost $10, is mathematically modeled as 4x(sub 1) + 6x(sub 2) + 5x (sub 3) = 10, where x(sub 1), x(sub 2), x(sub 3) are each cost of one mango, that of one banana, and that of one orange, respectively. All the information put together in a specified context, constitutes the physical problem and need not be all distinct. Some of these could be redundant, which cannot be readily identified by inspection. The resulting mathematical model will thus have equations corresponding to this redundant information and hence are linearly dependent and thus superfluous. Consequently, these equations once identified should be better pruned in the process of solving the system. The benefits are (i) less computation and hence less error and consequently a better quality of solution and (ii) reduced storage requirements. In literature, the pruning concept is not in vogue so far although it is most desirable. In a numerical linear system, the system could be slightly inconsistent or inconsistent of varying degree. If the system is too inconsistent, then we should fall back on to the physical problem (PP), check the correctness of the PP derived from the material universe, modify it, if necessary, and then check the corresponding mathematical model (MM) and correct it. In nature/material universe, inconsistency is completely nonexistent. If the MM becomes inconsistent, it could be due to error introduced by the concerned measuring device and/or due to assumptions made on the PP to obtain an MM which is relatively easily solvable or simply due to human error. No measuring device can usually measure a quantity with an accuracy greater that 0.005% or, equivalently with a relative error less than 0.005%. Hence measurement error is unavoidable in a numerical linear system when the quantities are continuous (or even discrete with extremely large number). Assumptions, though not desirable, are usually made when we find the problem sufficiently difficult to be solved within the available means/tools/resources and hence distort the PP and the corresponding MM. The error thus introduced in the system could (not always necessarily though) make the system somewhat inconsistent. If the inconsistency (contradiction) is too much then one should definitely not proceed to solve the system in terms of getting a least-squares solution or a minimum norm solution or the minimum-norm least-squares solution. All these solutions will be invariably of no real-world use. If, on the other hand, inconsistency is reasonably low, i.e. the system is near-consistent or, equivalently, has near-linearly-dependent rows, then the foregoing solutions are useful. Pruning in such a near-consistent system should be performed based on the desired accuracy and on the definition of near-linear dependence. In this article, we discuss pruning over various kinds of linear systems and strongly suggest its use as a pre-processor or as a part of an algorithm. Ideally pruning should (i) be a part of the solution process (algorithm) of the system, (ii) reduce both computational error and complexity of the process, and (iii) take into account the numerical zero defined in the context. These are precisely what we achieve through our proposed O(mn2) algorithm presented in Matlab, that uses a subprogram of solving a single linear equation and that has embedded in it the pruning.

Sen, Syamal K.↗
NASA NTRS2011

Should Pruning be a Pre-Processor of any Linear System?

There are many real-world problems whose mathematical models turn out to be linear systems Ax = b, where A is an m x n matrix. Each equation of the linear system is an information. An information, in a physical problem, such as 4 mangoes, 6 bananas, and 5 oranges cost $10, is mathematically modeled as an equation 4x(sub 1) + 6x(sub 2) + 5x(sub 3) = 10 , where x(sub 1), x(sub 2), x(sub 3) are each cost of one mango, that of one banana, and that of one orange, respectively. All the information put together in a specified context, constitutes the physical problem and need not be all distinct. Some of these could be redundant, which cannot be readily identified by inspection. The resulting mathematical model will thus have equations corresponding to this redundant information and hence are linearly dependent and thus superfluous. Consequently, these equations once identified should be better pruned in the process of solving the system. The benefits are (i) less computation and hence less error and consequently a better quality of solution and (ii) reduced storage requirements. In literature, the pruning concept is not in vogue so far although it is most desirable. It is assumed that at least one information, i.e. one equation is known to be correct and which will be our first equation. In a numerical linear system, the system could be slightly inconsistent or inconsistent of varying degree. If the system is too inconsistent, then we should fall back on to the physical problem (PP), check the correctness of the PP derived from the material universe, modify it, if necessary, and then check the corresponding mathematical model (MM) and correct it. In nature/material universe, inconsistency is completely nonexistent. If the MM becomes inconsistent, it could be due to error introduced by the concerned measuring device and/or due to assumptions made on the PP to obtain an MM which is relatively easily solvable or simply due to human error. No measuring device can usually measure a quantity with an accuracy greater that 0.005% or, equivalently with a relative error less than 0.005%. Hence measurement error is unavoidable in a numerical linear system when the quantities are continuous (or even discrete with extremely large number). Assumptions, though not desirable, are usually made when we find the problem sufficiently difficult to be solved within the available means/tools/resources and hence distort the PP and the corresponding MM. The . error thus introduced in the system could (not always necessarily though) make the system somewhat inconsistent. If the inconsistency (contradiction) is too much then one should definitely not proceed to solve the system in terms of getting a least-squares solution or the minimum-norm least-squares solution. All these solutions will be invariably of no real-world use. If, on the other hand, inconsistency is reasonably low, i.e. the system is near-consistent or, equivalently, has near-linearly-dependent rows, then the foregoing solutions are useful. Pruning in such a near-consistent system should be performed based on the desired accuracy and on the definition of near-linear dependence. In this article, we discuss pruning over various kinds of linear systems and strongly suggest its use as a pre-processor or as a part of an algorithm. Ideally pruning should (i) be a part of the solution process (algorithm) of the system, (ii) reduce both computational error and complexity of the process, and (iii) take into account the numerical zero defined in the context. These are precisely what we achieve through our proposed O(mn2) algorithm presented in Matlab, that uses a subprogram of solving a single linear equation and that has embedded in it the pruning.

Sen, Syamal K.↗
DOE OSTI2025

network-pruner (Neural network pruning analysis) [SWR-25-113]

This repository implements an iterative magnitude pruning algorithm for pruning neural networks in PyTorch. The pruning method involves gradually removing less significant weights from the model to achieve a specified sparsity, followed by fine-tuning the pruned model to recover performance.

Griffin, Kevin [National Renewable Energy Laborato↗
DOE OSTI2025

SymbolNet: neural symbolic regression with adaptive dynamic pruning for compression

Abstract Compact symbolic expressions have been shown to be more efficient than neural network (NN) models in terms of resource consumption and inference speed when implemented on custom hardware such as field-programmable gate arrays (FPGAs), while maintaining comparable accuracy (Tsoi et al 2024 EPJ Web Conf. 295 09036). These capabilities are highly valuable in environments with stringent computational resource constraints, such as high-energy physics experiments at the CERN Large Hadron Collider. However, finding compact expressions for high-dimensional datasets remains challenging due to the inherent limitations of genetic programming (GP), the search algorithm of most symbolic regression (SR) methods. Contrary to GP, the NN approach to SR offers scalability to high-dimensional inputs and leverages gradient methods for faster equation searching. Common ways of constraining expression complexity often involve multistage pruning with fine-tuning, which can result in significant performance loss. In this work, we propose S y m b o l N e t , a NN approach to SR specifically designed as a model compression technique, aimed at enabling low-latency inference for high-dimensional inputs on custom hardware such as FPGAs. This framework allows dynamic pruning of model weights, input features, and mathematical operators in a single training process, where both training loss and expression complexity are optimized simultaneously. We introduce a sparsity regularization term for each pruning type, which can adaptively adjust its strength, leading to convergence at a target sparsity ratio. Unlike most existing SR methods that struggle with datasets containing more than O ( 10 ) inputs, we demonstrate the effectiveness of our model on the LHC jet tagging task (16 inputs), MNIST (784 inputs), and SVHN (3072 inputs).

Tsoi, Ho Fung (ORCID:0000000225502184)↗
NASA NTRS1992

Recursive optimal pruning with applications to tree structured vector quantizers

A pruning algorithm of Chou et al. (1989) for designing optimal tree structures identifies only those codebooks which lie on the convex hull of the original codebook's operational distortion rate function. The authors introduce a modified version of the original algorithm, which identifies a large number of codebooks having minimum average distortion, under the constraint that, in each step, only modes having no descendents are removed from the tree. All codebooks generated by the original algorithm are also generated by this algorithm. The new algorithm generates a much larger number of codebooks in the middle- and low-rate regions. The additional codebooks permit operation near the codebook's operational distortion rate function without time sharing by choosing from the increased number of available bit rates. Despite the statistical mismatch which occurs when coding data outside the training sequence, these pruned codebooks retain their performance advantage over full search vector quantizers (VQs) for a large range of rates.

Kiang, Shei-Zein↗
DOE OSTI2024

Fast Iterative Multi-site Hosting Capacity Analysis for Distribution Systems With Search Space Pruning

Interconnection studies for distributed energy resources (DERs) is a time-intensive process, primarily due to the necessity of solving large number of power flow scenarios. Hosting capacity analysis (HCA) is a time-consuming aspect of interconnection studies that is divided into single-site HCA (SHCA) and multi-site HCA (MHCA). From a computational and understandable standpoint, the industry seeks iteration-based solutions for SHCA, although it doesn't maximize the total DER hosting capacity (DERHC) of the grid, as MHCA does. While non-iterative solutions are available for MHCA, they involve a trade-off between the modeling accuracy of the distribution system, solution quality, and ease of understanding. In this work, we present a fast iterative solution for MHCA, reducing computational complexity by eliminating the need to solve power flows for a large amount of search space, thus making iterative solutions feasible. This iterative approach guarantees both a global optimal solution with sufficient time and a fast, close-to-optimal solution through efficient search space pruning. It also easily integrates with existing utility HCA tools. The results are demonstrated on select locations in the IEEE-123 bus system for community-scale interconnection studies. We highlight the benefits of skipping the need to solve millions of power flows, all while maximizing the grid's total DERHC.

Guddanti, Kishan Prudhvi↗
NASA NTRS1998

Toward a More Robust Pruning Procedure for MLP Networks

Choosing a proper neural network architecture is a problem of great practical importance. Smaller models mean not only simpler designs but also lower variance for parameter estimation and network prediction. The widespread utilization of neural networks in modeling highlights an issue in human factors. The procedure of building neural models should find an appropriate level of model complexity in a more or less automatic fashion to make it less prone to human subjectivity. In this paper we present a Singular Value Decomposition based node elimination technique and enhanced implementation of the Optimal Brain Surgeon algorithm. Combining both methods creates a powerful pruning engine that can be used for tuning feedforward connectionist models. The performance of the proposed method is demonstrated by adjusting the structure of a multi-input multi-output model used to calibrate a six-component wind tunnel strain gage.

Stepniewski, Slawomir W.↗
NASA NTRS2003

Mining Distance Based Outliers in Near Linear Time with Randomization and a Simple Pruning Rule

Defining outliers by their distance to neighboring examples is a popular approach to finding unusual examples in a data set. Recently, much work has been conducted with the goal of finding fast algorithms for this task. We show that a simple nested loop algorithm that in the worst case is quadratic can give near linear time performance when the data is in random order and a simple pruning rule is used. We test our algorithm on real high-dimensional data sets with millions of examples and show that the near linear scaling holds over several orders of magnitude. Our average case analysis suggests that much of the efficiency is because the time to process non-outliers, which are the majority of examples, does not depend on the size of the data set.

Bay, Stephen D.↗
DOE OSTI2026

Locally purified maximally mixed states at scale: Entanglement pruning and symmetries

Locally Purified Density Operators (LPDOs) are state-of-the-art tensor network ansatze candidates that efficiently represent mixed quantum states at scale. However, given their non-uniqueness, their representational complexity is generally sub-optimal in practical computations. Here, in this work we perform a comprehensive numerical and analytical analysis and resolve this issue in the experimentally relevant limit where noise depolarizes the density operator into a maximally mixed state. To resolve the sub-optimality issue, we analyze two numerical tools, one analytic method, and detail the relations between them. The numerical tools used are fidelity-preserving truncations and isometric gauge transformations leveraging Riemannian optimizations over entropic objective functions. In addition, by invoking the injectivity and symmetry constraints of the maximally mixed LPDO, we also present analytical closed-form expressions for the disentangler and discuss their relation to numerical optimizers. Further, away from the maximally mixed state, our simulations highlight how the truncation threshold smoothly interpolate, as a function of depolarization, between established matrix product results and our new results. Our work shows how, by minimizing the resources required to represent key states of practical interest in experiment, the efficiency of tensor network algorithms can be substantially increased. This paves the path for uncovering tensor network’s fundamental scalability limits and latent potential in representing the wide locus of mixed quantum states that are accessible on near-term quantum devices.

Gangapuram, Amit Jamadagni [Oak Ridge National Lab↗
NASA NTRS1991

Surface reconstruction from scattered data through pruning of unstructured grids

This paper describes an algorithm for reconstructing a surface from a randomly digitized object. Scan data (treated as a cloud of points) is first tesselated out to its convex hull using Delaunay triangulation. The line-of-sight between each surface point and the scanning device is traversed, and any tetrahedra which are pierced by it are removed. The remaining tetrahedra form an approximate solid model of the scanned object. Due to the inherently limited resolution of any scan, this algorithm requires two additional procedures to produce a smooth, polyhedral surface: one process removes long, narrow tetrahedra which span indentations in the surface between digitized points; the other smooths sharp edges. The results for a moderately resolved sample body and a highly resolved aircraft are displayed.

Maksymiuk, C. M.↗
NASA NTRS1993

Pruning the Lyman alpha forest of Q1331+170

A multitude of absorption lines seen shortward of QSO Ly-alpha emission, that cannot be traced to heavy element absorption systems, are assumed to be Ly-alpha lines arising in intervening clouds. Studies of these Ly-alpha clouds, typically done at 1 A or lower resolution, have shown N(H1) approximately equal to 10(exp 13) - 10(exp 17) sq cm and b approximately equal to 35 km/s. Sargent et al 1980, on the basis of a flat pair velocity correlation function (PVCF), argued that these clouds are intergalactic. But Crotts 1989 showed that the strong Ly-alpha lines are spatially clustered. High resolution studies of Webb 1987 and Rauch et al 1992 also report some evidence for weak clustering, but overall such high resolution studies have been rare. A study of the Ly-alpha forest of Q1331 + 170 over x(sub abs) = 1.60 - 2.19 based on 18 km/s resolution data at S/N approximately equal to 15, with metal-line deblending incorporated.

Kulkarni, Varsha P.↗
NASA NTRS1997

Locating Geometric Primitives by Pruning the Parameter Space

This paper examines the extraction of geometric primitives from two- and three-dimensional image data. The geometric primitives are represented by parametric manifolds in the image space, such as circles, planes, and cylinders.

geometric primitives image data parameter space im↗
NASA NTRS1997

Pointer Adaptation and Pruning of Min-Max Fuzzy Inference and Estimation

A new technique for adaptation of fuzzy membership functions in a fuzzy inference system is proposed. The pointer technique relies upon the isolation of the specific membership functions that contributed to the final decision, followed by the updating of these functions' parameters using steepset descent.

Fuzzy↗
DOE OSTI2026

Mitigating Algorithmic Bias in Cancer Site Classification Models

Purpose Integrating artificial intelligence in cancer diagnostics has improved tumor classification beyond rule-based systems. Despite these advancements, these models may still encode demographic biases. We conducted a large-scale, applied bias-probing study of a deep learning–based cancer site classifier to quantify race information encoded in document embeddings. We then evaluated how performance changes when race-correlated embedding dimensions are removed in a post-training sensitivity analysis. Methods The cancer site classifier was trained using 3.5 million electronic cancer pathology reports from six of the National Cancer Institute's SEER registries. We trained a hierarchical self-attention network to generate 400-dimensional document embeddings. These embeddings were used to train two downstream, gradient-boosted decision tree classifiers: one to classify the cancer sites and another to predict racial categories. We identified overlapping features by intersecting the top 50 feature-importance rankings from the site and race models and computed their cumulative feature importance in each model. As a post hoc sensitivity analysis, we progressively pruned these overlapping dimensions, retrained the site model, and compared overall macro-F1 and accuracy, race-stratified macro-F1, and group fairness metrics on the basis of demographic parity and equalized odds before and after pruning. Results The analysis revealed minimal feature overlap between the cancer site and race prediction models, and the cumulative importance scores indicated a negligible influence of racial information on clinical predictions. Post-training pruning of overlapping features did not compromise the models' diagnostic accuracy, with a 0.07% loss in accuracy. Conclusion Our findings demonstrate that HiSAN-generated embeddings from SEER data can be used effectively in cancer site classification without significant demographic bias influencing the outcomes. Post-training pruning therefore functions as a practical audit and sensitivity check.

Shivanna, Abhishek [ORNL] (ORCID:0009000665228593)↗
NASA NTRS1992

Purposive discovery of operations

The Generate, Prune & Prove (GPP) methodology for discovering definitions of mathematical operators is introduced. GPP is a task within the IL exploration discovery system. We developed GPP for use in the discovery of mathematical operators with a wider class of representations than was possible with the previous methods by Lenat and by Shen. GPP utilizes the purpose for which an operator is created to prune the possible definitions. The relevant search spaces are immense and there exists insufficient information for a complete evaluation of the purpose constraint, so it is necessary to perform a partial evaluation of the purpose (i.e., pruning) constraint. The constraint is first transformed so that it is operational with respect to the partial information, and then it is applied to examples in order to test the generated candidates for an operator's definition. In the GPP process, once a candidate definition survives this empirical prune, it is passed on to a theorem prover for formal verification. We describe the application of this methodology to the (re)discovery of the definition of multiplication for Conway numbers, a discovery which is difficult for human mathematicians. We successfully model this discovery process utilizing information which was reasonably available at the time of Conway's original discovery. As part of this discovery process, we reduce the size of the search space from a computationally intractable size to 3468 elements.

Sims, Michael H.↗
DOE OSTI2026

Unsupervised atomic data mining via multi-kernel graph autoencoders for machine learning force fields

Constructing a chemically diverse dataset while avoiding sampling bias is critical to training efficient and generalizable force fields. However, in computational chemistry and materials science, many common dataset generation techniques are prone to oversampling regions of the potential energy surface. Furthermore, these regions can be difficult to identify and isolate from each other or may not align well with human intuition, making it challenging to systematically remove bias in the dataset. While traditional clustering and pruning (down-sampling) approaches can be useful for this, they can often lead to information loss or a failure to properly identify distinct regions of the potential energy surface due to difficulties associated with the high dimensionality of atomic descriptors. In this work, we introduce the Multi-kernel Edge Attention-based Graph Autoencoder (MEAGraph) model, an unsupervised approach for analyzing atomic datasets. MEAGraph combines multiple linear kernel transformations with attention-based message passing to capture geometric sensitivity and enable effective dataset pruning without relying on labels or extensive training. Demonstrated applications on niobium, tantalum, and iron datasets show that MEAGraph efficiently groups similar atomic environments, allowing for the use of basic pruning techniques for removing sampling bias. This approach provides an effective method for representation learning and clustering that can be used for data analysis, outlier detection, and dataset optimization.

Materials science↗