SEARCH · Search NASA
Results for “pruning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Fine-Pruning: A biologically inspired algorithm for personalization of machine learning models
Not Available
Locally purified maximally mixed states at scale: Entanglement pruning and symmetries
Locally Purified Density Operators (LPDOs) are state-of-the-art tensor network ansatze candidates that efficiently represent mixed quantum states at scale. However, given their non-uniqueness, their representational complexity is generally sub-optimal in practical computations. Here, in this work we perform a comprehensive numerical and analytical analysis and resolve this issue in the experimentally relevant limit where noise depolarizes the density operator into a maximally mixed state. To resolve the sub-optimality issue, we analyze two numerical tools, one analytic method, and detail the relations between them. The numerical tools used are fidelity-preserving truncations and isometric gauge transformations leveraging Riemannian optimizations over entropic objective functions. In addition, by invoking the injectivity and symmetry constraints of the maximally mixed LPDO, we also present analytical closed-form expressions for the disentangler and discuss their relation to numerical optimizers. Further, away from the maximally mixed state, our simulations highlight how the truncation threshold smoothly interpolate, as a function of depolarization, between established matrix product results and our new results. Our work shows how, by minimizing the resources required to represent key states of practical interest in experiment, the efficiency of tensor network algorithms can be substantially increased. This paves the path for uncovering tensor network’s fundamental scalability limits and latent potential in representing the wide locus of mixed quantum states that are accessible on near-term quantum devices.
Adaptive pruning-based optimization of parameterized quantum circuits
Abstract Variational hybrid quantum–classical algorithms are powerful tools to maximize the use of noisy intermediate-scale quantum devices. While past studies have developed powerful and expressive ansatze, their near-term applications have been limited by the difficulty of optimizing in the vast parameter space. In this work, we propose a heuristic optimization strategy for such ansatze used in variational quantum algorithms, which we call ‘parameter-efficient circuit training (PECT)’. Instead of optimizing all of the ansatz parameters at once, PECT launches a sequence of variational algorithms, in which each iteration of the algorithm activates and optimizes a subset of the total parameter set. To update the parameter subset between iterations, we adapt the Dynamic Sparse Reparameterization scheme which was originally proposed for training deep convolutional neural networks. We demonstrate PECT for the Variational Quantum Eigensolver, in which we benchmark unitary coupled-cluster ansatze including UCCSD and k -UpCCGSD, as well as the Low-Depth Circuit Ansatz (LDCA), to estimate ground state energies of molecular systems. We additionally use a layerwise variant of PECT to optimize a hardware-efficient circuit for the Sycamore processor to estimate the ground state energy densities of the one-dimensional Fermi-Hubbard model. From our numerical data, we find that PECT can enable optimizations of certain ansatze that were previously difficult to converge and more generally can improve the performance of variational algorithms by reducing the optimization runtime and/or the depth of circuits that encode the solution candidate(s).
Scalable Contour Tree Computation by Data Parallel Peak Pruning
As data sets grow to exascale, automated data analysis and visualization are increasingly important, to intermediate human understanding and to reduce demands on disk storage via in situ analysis. Trends in architecture of high performance computing systems necessitate analysis algorithms to make effective use of combinations of massively multicore and distributed systems. One of the principal analytic tools is the contour tree, which analyses relationships between contours to identify features of more than local importance. Unfortunately, the predominant algorithms for computing the contour tree are explicitly serial, and founded on serial metaphors, which has limited the scalability of this form of analysis. While there is some work on distributed contour tree computation, and separately on hybrid GPU-CPU computation, there is no efficient algorithm with strong formal guarantees on performance allied with fast practical performance. Furthermore, we report the first shared SMP algorithm for fully parallel contour tree computation, with formal guarantees of O(lg V lg t) parallel steps and O(V lg V) work for data with V samples and t contour tree supernodes, and implementations with more than 30× parallel speed up on both CPU using TBB and GPU using Thrust and up 70× speed up compared to the serial sweep and merge algorithm.
Hatchet: Pruning the Overgrowth in Parallel Profiles
Explore the source record for details and available documents.
PruneJuice: Pruning Trillion-edge Graphsto a Precise Pattern-Matching Solution
Explore the source record for details and available documents.
Reinforcement Learning Policy Performance Impacts of Pruning Quantization and Compression.
Abstract not provided.
Investigating Channel Pruning through Structural Redundancy Reduction - A Statistical Study
Explore the source record for details and available documents.
The Generalization-Stability Tradeoff In Neural Network Pruning
Explore the source record for details and available documents.
Calibrate and Prune: Improving Reliability of Lottery Tickets Through Prediction Calibration
Explore the source record for details and available documents.
PQV-Mobile: A Combined Pruning and Quantization Toolkit to Optimize Vision Transformers for Mobile Applications
Explore the source record for details and available documents.
CSB-RNN: A Faster-Than-Realtime RNN Acceleration Framework with Compressed Structured Blocks
Recurrent Neural Networks (RNN) is widely applied to temporal sequence analysis, where real-time performance is usually in demand. However, RNN suffers a heavy computational workload as the model comes with a large weight matrix. To alleviate the pain, model compression (pruning) schemes have been proposed for RNN that pruning the redundant (near-zero) weight-values. On the one hand, the non-structured pruning methods achieve a considerable pruning rate while bringing the computational irregularity, which is un-friendly to parallel-hardware. On the other hand, the existing structured pruning methods consider the hardware parallelism; However, they suffer a poor pruning rate due to the restrict constraints on pruning structure. This paper presents CSB-RNN, an optimized full-stack RNN framework with the novel compressed structured block (CSB) technique. The CSB-pruned RNN model comes with both fine-granularity that benefits the pruning rate and regular structure that facilitates the hardware-parallelism. Further, we propose a novel hardware architecture for inferencing the CSB-pruned model. Different from conventional parallel hardware, this architecture solves the block-workload imbalance issue and achieves an over 95% hardware utilization. With the experiments on 10 RNN models in 5 application domains, the CSB-RNN realizes 7×-20× lossless compression and up to 50× acceptable lossy-compression, which is 2×-7× to the prior art. With the addition of the novel hardware, the compressed-RNN inference reaches a super real-time latency of 10-400µs with FPGA implementation.
Local rules for fabricating allosteric networks
Mechanical properties of disordered networks can be significantly tailored by modifying a small fraction of their bonds. This procedure has been used to design and build mechanical metamaterials with a variety of responses. A long-range “allosteric” response, where a localized input strain at one site gives rise to a localized output strain at a distant site, has been of particular interest. This work presents an approach to incorporating allosteric responses in experimental systems by pruning disordered networks in situ. Previous work has relied on computer simulations to design and predict the response of such systems using a cost function where the response of the entire network to each bond removal is used at each step to determine which bond to prune. It is not feasible to follow such a design protocol in experiments where one has access only to local response at each site. This paper presents design algorithms that allow determination of what bonds to prune based purely on the local forces in the network without employing a cost function; using only local information, allosteric networks are designed in simulations and then built out of real materials. The results show that some pruning strategies work better than others when translated into an experimental system. A method is presented to measure local stresses experimentally in disordered networks. This approach is then used to implement pruning methods to design desired responses in situ. Results from these experiments confirm that the pruning methods are robust and work in a real laboratory material.
O3BNN-R: An Out-Of-Order Architecture for High-Performance and Regularized BNN Inference
Binarized Neural Networks (BNN) have drawn tremendous attention due to significantly reduced computational complexity and memory demand. They have especially shown great potential in cost- and power-restricted domains, such as IoT and smart edge-devices, where reaching a certain accuracy bar is often sufficient, and real-time is highly desired.In this work, we demonstrate that the highly-condensed BNN model can be shrunk significantly further by dynamically pruning irregular redundant edges. Based on two new observations on BNN-specific properties, an out-of-order (OoO) architecture – O3BNN-R, can curtail edge evaluation in cases where the binary output of a neuron can be determined early. Similar to Instruction-Level-Parallelism(ILP), these fine-grained, irregular, runtime pruning opportunities are traditionally presumed to be difficult to exploit. In order to increase the pruning opportunities, we also optimize the training process by adding 2 regularization items in the loss function (1) for pooling pruning and (2) for threshold pruning. We evaluate our design on an FPGA platform using three well-known networks, including VggNet-16, AlexNet for ImageNet, and a VGG-like network for Cifar-10.
Towards Efficient Convolutional Neural Networks Through Low-Error Filter Saliency Estimation
Filter saliency based channel pruning is a state-of-the-art method for deep convolutional neural network compression and acceleration. This channel pruning method ranks the importance of individual filter by estimating its impact of each filter’s removal on the training loss, and then remove the least important filters and fine-tune the remnant network. In this work, we propose a systematic channel pruning method that significantly reduces the estimation error of filter saliency. Different from existing approaches, our method largely reduces the magnitude of parameters in a network by introducing alternating direction method of multipliers (ADMM) into the pre-training procedure. Therefore, the estimation of filter saliency based on Taylor expansion is significantly improved. Extensive experiments with various benchmark network architectures and datasets demonstrate that the proposed method has a much improved unimportant filter selection capability and outperform state-of-the-art channel pruning method.
Mitigating Algorithmic Bias in Cancer Site Classification Models
Purpose Integrating artificial intelligence in cancer diagnostics has improved tumor classification beyond rule-based systems. Despite these advancements, these models may still encode demographic biases. We conducted a large-scale, applied bias-probing study of a deep learning–based cancer site classifier to quantify race information encoded in document embeddings. We then evaluated how performance changes when race-correlated embedding dimensions are removed in a post-training sensitivity analysis. Methods The cancer site classifier was trained using 3.5 million electronic cancer pathology reports from six of the National Cancer Institute's SEER registries. We trained a hierarchical self-attention network to generate 400-dimensional document embeddings. These embeddings were used to train two downstream, gradient-boosted decision tree classifiers: one to classify the cancer sites and another to predict racial categories. We identified overlapping features by intersecting the top 50 feature-importance rankings from the site and race models and computed their cumulative feature importance in each model. As a post hoc sensitivity analysis, we progressively pruned these overlapping dimensions, retrained the site model, and compared overall macro-F1 and accuracy, race-stratified macro-F1, and group fairness metrics on the basis of demographic parity and equalized odds before and after pruning. Results The analysis revealed minimal feature overlap between the cancer site and race prediction models, and the cumulative importance scores indicated a negligible influence of racial information on clinical predictions. Post-training pruning of overlapping features did not compromise the models' diagnostic accuracy, with a 0.07% loss in accuracy. Conclusion Our findings demonstrate that HiSAN-generated embeddings from SEER data can be used effectively in cancer site classification without significant demographic bias influencing the outcomes. Post-training pruning therefore functions as a practical audit and sensitivity check.