Search NASA⌕ Search

SEARCH · Search NASA

Results for “Ranking methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Correlated purification for restoring 𝑁-representability in quantum simulation

Experimentally measured reduced density matrices (RDMs) often violate constraints that ensure they represent N-electron states—known as N-representability conditions—because of statistical and hardware noise. In this work, we present a correlated purification framework based on semidefinite programming to restore the accuracy of a noisy, unphysical two-electron RDM (2-RDM). The method performs a bi-objective optimization that minimizes both the many-electron energy and the nuclear norm of the correction to the measured 2-RDM. The nuclear norm, often employed in matrix completion, promotes low-rank corrections, while the energy term acts as a regularization term that can improve the purity of the ground state. While the method is particularly effective for ground states, it can also be applied to excited and nonstationary states by decreasing the weight of the energy relative to the error norm. In an application to fermionic shadow tomography of large hydrogen chains, correlated purification yields substantial reductions in both energy and 2-RDM error, achieving chemical accuracy across dissociation curves. This framework provides a robust strategy for tomography in many-body quantum simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Assessing the predictive impact of factor fixing with an adaptive uncertainty-based approach .

Despite widespread use of factor fixing in environmental modeling, its effect on model predictions has received little attention and is instead commonly presumed to be negligible. We propose a proof-of-concept adaptive method for systematically investigating the impact of factor fixing. The method uses Global Sensitivity Analysis methods to identify groups of sensitive parameters, then quantifies which groups can be safely fixed at nominal values without exceeding a maximum acceptable error, demonstrated using the 21-dimensional Sobol’ G-function. Furthermore, three error measures are considered for quantities of interest, namely Relative Mean Absolute Error, Pearson Product-Moment Correlation and Relative Variance. Results demonstrate that factor fixing may cause large errors in the model results unexpectedly, when preliminary analysis suggests otherwise, and that the default value selected affects the number of factors to fix. To improve the applicability and methodological development of factor fixing, a new research agenda encompassing five opportunities is discussed for further attention.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Quantum Tensor-Product Decomposition from Choi-State Tomography

The Schmidt decomposition is the go-to tool for measuring bipartite entanglement of pure quantum states. Similarly, it is possible to study the entangling features of a quantum operation using its operator-Schmidt or tensor-product decomposition. While quantum technological implementations of the former are thoroughly studied, entangling properties on the operator level are harder to extract in the quantum computational framework because of the exponential nature of sample complexity. Here, we present an algorithm for unbalanced partitions into a small subsystem and a large one (the environment) to compute the tensor-product decomposition of a unitary the effect of which on the small subsystem is captured in classical memory, while the effect on the environment is accessible as a quantum resource. This quantum algorithm may be used to make predictions about operator nonlocality and effective open quantum dynamics on a subsystem, as well as for finding low-rank approximations and low-depth compilations of quantum circuit unitaries. We demonstrate the method and its applications on a time-evolution unitary of an isotropic Heisenberg model in two dimensions. Published by the American Physical Society 2024

Mansuroglu, Refik (ORCID:000000017352513X)↗
DOE OSTI2020

Economic Extraction and Recovery of REEs and Production of Clean Value-Added Products from Low-Rank Coal Fly Ash

The University of North Dakota (UND) Energy & Environmental Research Center (EERC) teamed with Pacific Northwest National Laboratory (PNNL) , the North Dakota Industrial Commission Lignite Research Program, and commercial partners Basin Electric Power Cooperative, Great River Energy, and Southern Company to execute a Cooperative Agreement funded by the U.S. Department of Energy’s (DOE’s) National Energy Technology Laboratory (NETL) focused on identifying unique pathways and pretreatments to extract rare-earth elements (REEs) from low-rank coal (LRC) ash in a more economical and environmentally benign manner than current methods. The EERC drew upon the unique and deep knowledge of the team regarding LRC geochemistry, ash transformation mechanisms during combustion, and fly ash chemistry in the approach to understand the portioning of REEs in LRC ash. A comprehensive characterization effort was undertaken to fully establish the form, associations, and partitioning of the REEs and other elements/minerals of interest in fly ash, as well as the ash chemistry, mineralogy, and morphology. Using this information, the team tested methods of REE extraction to take advantage of the unique and advantageous properties of LRC ash. Multiple complementary approaches were evaluated that combined both novel and proven methods to develop an extractive process for REEs and other high-value metals from LRC ash that provide technological and economic progress on the current state of the art. Methodologies were also evaluated for the potential to synergistically remove toxic metals while extracting other high-value metals from the ash, generating a clean, safe ash material as a value-added product. This would eliminate waste and offer significant additional revenue potential. Based on laboratory-scale testing, an initial high-level technical and economic analysis to estimate operating expenses and product revenues to guide future phases of technology development was implemented. By focusing on minimizing process steps and consumptive use of extraction solutions, a tunable, economically viable process for REE extraction from LRC ash was developed. The tunable process has the ability to be adjusted to adapt to the differing ash chemical and physical properties that are characteristic of LRC ash. This is key to deployment as a commercial process. In demonstrating the process, the EERC was able to illustrate mixed REE concentration greater than the goal of 2 wt%. High-level economic analysis of the process with LRC ash of moderate REE levels demonstrated the potential of the process, but recovery of just the REEs was insufficient to cover operational costs. With the additional extraction of other high-value metals found in LRC ash, the process economics approach would break even, and it is anticipated that with process optimization, the developed process could potentially be economically sound. The same economic analysis performed utilizing LRC ash with high levels of REEs showed the recovery of the REEs alone would make the process economically viable even without further optimization.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2022

Economic Extraction and Recovery of REEs and Production of Clean Value-Added Products from Low-Rank Coal Fly Ash

TechConnect World Innovation Conference & Expo, Washington, DC, June 13–15, 2022. The University of North Dakota (UND) Energy & Environmental Research Center (EERC) teamed with Pacific Northwest National Laboratory; the North Dakota Industrial Commission Lignite Research Program; and commercial partners Basin Electric Power Cooperative, Great River Energy, and Southern Company to execute a Cooperative Agreement funded by the U.S. Department of Energy’s National Energy Technology Laboratory (NETL) focused on identifying unique pathways and pretreatments to extract rare-earth elements (REEs) from low-rank coal (LRC) ash in a more economical and environmentally benign manner than current methods. By focusing on minimizing process steps and consumptive use of extraction solutions, a tunable, economically viable process for REE extraction from LRC ash was developed.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2020

Applicability of the Delplot method for the determination of catalytic reaction sequences: Hydrodeoxygenation of $\gamma$-valerolactone on Ni 2 P/MCM-41

Contact-time measurements based on sites are used to investigate the reaction sequence of the hydrodeoxygenation of γ-valerolactone (GVL) on Ni 2 P/MCM-41. Here the study is carried out at 300 °C and 0.5 MPa with site times of 0.29–6.1 s. A reaction network is proposed in which pentanoic acid and 2-methyltetrahydrofuran (2-MTHF) are formed in parallel initially and pentanal, butane and pentane are formed subsequently. A first-rank Delplot analysis (selectivity) gives a good fit, but higher-rank Delplots do not. The lack of applicability is explained by the magnitude of the rate constants, in which the rate constant for one of the later steps (the consumption of pentanal) is substantially larger than those for the preceding steps. For this reason, pentanal demonstrated the characteristics of a primary product and consequently affected the discrimination of higher rank products. A simulation of a model sequence with assumed rate constants confirms the reason why the Delplot method does not work for this reaction. Since the rate constants cannot be determined a priori to be of close magnitude, the Delplot method cannot be generally used to ascertain reaction sequences.

42 ENGINEERING↗
DOE OSTI2022

Neutron diffusion calculation in heterogeneous geometry based on local/global iteration using proper orthogonal decomposition

This study newly proposes a heterogeneous core calculation method based on local/global iteration using proper orthogonal decomposition (POD). By using the singular value decomposition (SVD) and the low-rank approximation, appropriate POD bases for expanding the neutron flux can be obtained from snapshot data of the neutron flux obtained by fine mesh calculations. By projection using the POD bases, the dimension of the target equation (e.g., discretized neutron diffusion equation) can be dramatically reduced. In the proposed method, POD is effectively applied to each single assembly calculation (local calculation). Furthermore, using the local/global iteration, the effective neutron multiplication factor and the neutron flux distribution in the whole core geometry can be obtained by combining the numerical results of the local calculation for each fuel assembly and the global calculation for the whole core. As a feasibility study, the proposed method is applied to a one-dimensional heterogeneous core analysis, and the accuracy is investigated by changing the total number of POD bases. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Sylvester-preconditioned adaptive-rank implicit time integrators for advection-diffusion equations with variable coefficients

Here, we consider the adaptive-rank integration of multi-dimensional time-dependent advection-diffusion partial differential equations (PDEs) with variable coefficients. We employ a standard finite-difference method for spatial discretization coupled with high-order diagonally implicit Runge-Kutta temporal schemes. The discrete equation is a generalized Sylvester equation (GSE), which we solve with a projection-based adaptive-rank algorithm structured around two key strategies: (i) constructing dimension-wise subspaces using a novel atypical extended Krylov strategy, and (ii) efficiently solving the basis coefficient matrix with a preconditioned GMRES solver. The low-rank decomposition is performed in 2D using SVD and with high-order SVD (HOSVD) in 3D to represent the tensor in a compressed Tucker format. For d-dimensional problems (here, d = 2 or 3), the computational complexity and memory storage of the approach are found numerically to scale as and $\mathscr{O}(Nr^2) + \mathscr{O} (r^{d+1})$ and $\mathscr{O}(Nr) + \mathscr{O} (r^{d})$, respectively, with the one-dimensional resolution and the maximal rank during the Krylov iteration (which we find to be largely independent of on our numerical examples). We present numerical examples that illustrate the advertised properties of the algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

OpenSn: A massively parallel, open-source simulation environment for discrete ordinates radiation transport

OpenSn is an open-source, massively parallel deterministic radiation transport code for solving the discrete-ordinates ( S N ) form of the Boltzmann transport equation on unstructured, arbitrary polyhedral meshes. It supports high-fidelity simulations involving steady-state, eigenvalue, and adjoint problems for neutral particles (e.g., neutrons, photons, multi-particles), using the multigroup approximation in energy. OpenSn combines angular discretization via discrete ordinates with a discontinuous Galerkin finite element method (DGFEM) in space, enabling accurate resolution of transport physics on arbitrary polyhedral cells, included locally refined spatial grids. It includes multiple angular quadrature types, including locally refined angular quadratures. Written in modern C++ with a Python API, OpenSn runs efficiently on platforms ranging from laptops to supercomputers. The transport sweep algorithm is implemented using a task-based, directed-acyclic-graph (DAG) approach for each angle and supports asynchronous parallelism across thousands of MPI ranks. Group-set aggregation improves compute intensity, and synthetic acceleration techniques (e.g., diffusion synthetic acceleration, second-moment method) enhance solver convergence. OpenSn has been verified on reactor physics problems and demonstrated excellent weak and strong scaling performance on more than 32,768 processes, making it a versatile and robust platform for large-scale transport simulations in complex geometries.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Laboratory-Scale Coal-Derived Graphene Process (Final Report)

The Energy & Environmental Research Center (EERC) conducted a laboratory-scale coal-derived graphene (CDG) project focused on developing a technological process for making graphene from four U.S. domestic coal or coal wastes, including lignite from North Dakota, subbituminous coal from Wyoming, bituminous coal from Utah, and anthracite from Pennsylvania. The project was divided into two performance or budget periods (BPs), with BP1 comprising the up-front laboratory experiments to make graphene materials from coal beginning on May 1, 2020, to April 30, 2022. BP2 was conducted from May 1, 2022, to April 30, 2023, and was focused on analyzing the CDG process economic feasibility and the technical gaps for technological scale-up and commercialization. During this project, a few different coal-derived high-value products have been demonstrated, including graphite, graphene oxide (GO), reduced graphene oxide (rGO), and graphene quantum dots (GQDs). A new graphite microstructure was discovered and named “croissant graphite” because of the exterior morphological and textural resemblance to croissant food items sold in commercial groceries stores. The new graphite structure and the associated preparation from coal or coal waste feedstocks has been the subject of a U.S. patent application. The systematic experimental processes involving coal cleaning, upgrading, and conversion to high-value carbon products culminated into a developed upgraded coal-to-products (UCP) technology that is being pursued for potential fast-track commercialization, if funding is available. It is envisioned that commercialization of the UCP technology would increase consumption of U.S. domestic coals or coal wastes to make environmentally sustainable high-value products for the electronics industry, high-energy-storage applications, and clean energy technologies such as electric vehicle (EV) lithium-ion batteries (LIBs), for which graphite has become a critical mineral commodity. Croissant graphite microstructures, when observed by field emission scanning electron microscopy (FESEM), display wavy surface morphology and often grow from a base that is made of graphitized particles with honeycomb-like layers, which are believed to be graphene layers. While more studies are needed to fully ascertain the mechanisms of the croissant graphite microstructure formation, it is postulated that their growth may begin from curling of the graphene sheets into ribbon-like structures, and continuous growth and densification of the ribbon-like structures forms croissant microstructures. Additional studies are ongoing to evaluate the electrochemical performance of croissant graphite for LIB applications and to determine the experimental conditions necessary to tune on/off croissant formation so that it can be either optimized or suppressed depending on performance evaluation results. In addition to the discovery of croissant graphite, the graphitization process from the four coal ranks in general was successful. X-ray diffraction (XRD) analysis showed that the degree of graphitization (DoG) ranged from 12% to 80% in an early sample set, and further optimization on lignite coal produces a DoG of about 92%, which was spectacular to see as lignite is the lowest-rank coal. Thus, it is expected that the graphitization performance for higher-rank coals will be similar or better when optimized as well. The coal-derived graphite was used to make GO and rGO. Analytical characterization, e.g., by methods such as Raman spectroscopy, XRD, Fourier transform infrared (FTIR) spectroscopy and FESEM, showed that the sequence of converting the coal to graphite, exfoliating it to GO, and then chemically reducing the GO to rGO was successful. Although coal naturally contains aromatic compounds and some relatively small-sized condensed aromatic units, it does not contain graphene sheets. In the UCP process, the aromatic domains in the coals, particularly low-rank coals, are concentrated and condensed further into graphene sheets, which are ordered into a 3D stack during graphitization. The synthesized graphite is then unpacked by methods such as exfoliation to various graphene products. GQDs were synthesized from all four coal types, and their optical properties were demonstrated to be tunable by the coal precursor preprocessing treatments. In all four coal types, enhanced optical properties were observed for the produced GDQs with incremental improvements made to the coal precursors. GQDs produced from raw coal samples displayed lower ultraviolet–visible (UV–Vis) spectroscopy absorbance intensity compared to those obtained from cleaned and upgraded coal residues. The photoluminescence (PL) intensities also varied with pretreatment conditions and with the concentration of GQDs in aqueous solutions. GQDs obtained from anthracite show longer emission wavelengths and can be excited by visible light as opposed to GQDs derived from the other coal ranks. UV fluorescence 3D maps and spectra revealed that the emission wavelength at which the GQDs solutions display the highest intensity was slightly redshifted based on the coal precursor pretreatments. In low-rank coal (lignite and subbituminous) samples, two clusters were observed in the maps for GQDs, which may suggest that there are potentially two types of fluorophores in solution or two main size populations. The ability to tune the properties of GQDs based on processing methods can be exploited to make GQDs for various optical display or optoelectronics applications. The results also highlight the importance of removing coal-borne impurities to improve the quality of the coal precursor for preparation of graphene products. Coal and/or coal wastes preprocessing methods were developed and applied to clean and upgrade the coal precursors prior to graphitization and subsequent conversion to graphene products. The preprocessing methods involve high specific-gravity separations, mineral acid cleaning (no hydrofluoric acid), and subsequent upgrading by reducing the coal-borne heteroatom (nitrogen, sulfur, and oxygen) content using proprietary chemical agents. Analytical characterization revealed that the preprocessing steps were successful, with ash reductions that range from 38% to 80% and residual ash content that was below the 5 wt% initial target. Based on proximate and ultimate analysis, the heteroatom reduction reactions produced upgraded coal residues with the oxygen content reduced by 8% to 24%, with additional reductions in the nitrogen and sulfur contents. An initial assessment of the waste streams from the UCP process shows very small to negligible environmental impact due to CO 2 , NO x , and SO x because most process steps are performed under inert atmosphere with argon. Consequently, reactive oxygen environments that tend to create these species are avoided. The inorganic and potentially hazardous species are released into aqueous waste streams that are easy to handle for proper disposal. The liquid waste streams were found to contain low-level concentrations of rare-earth elements (REEs), which could be concentrated and recovered as value-added by-products. Additionally, the volatile and gaseous fractions from carbonization and heat treatment contain useful organic compounds that can also be recovered as potential valuable by-products. Thus, the UCP technology is considered an environmentally sustainable and promising emerging technology for making high-value products from coal and coal wastes, with potential additional value-added by-products. Analysis of potential markets for the coal-derived carbon products shows a strong demand in both niche market sectors and across a wide variety of other industrial sectors. Graphite is currently considered a critical mineral commodity that has a large and growing demand in the LIB industry for EV applications. Based on data from Fortune Business Insights (2022) and Marketwatch (2023) reports, the average global graphite market is projected to reach about 33 billion by 2028, growing at a compound annual growth rate (CAGR) of about 7%, with much of this growth expected to be in the LIB industry. GO and rGO have strong market potentials in various application areas, such as coatings for anticorrosion, anti-icing, and antimicrobial protection, thermal barriers, wear resistance, sensors, additive manufacturing such as 3D inks, and others. GQDs are the emerging key player in the bioimaging, photovoltaics, and light-emitting diodes (LEDs) applications, with the potential to replace traditional semiconductor quantum dots (SQDs), which are based on metallic systems that are more toxic and more expensive. Biomedical applications of GQDs are becoming more attractive because of low to no toxicity and extremely low cost compared to SQDs. The major challenges for scale-up and commercialization of coal-derived carbon products such as graphene vary from the inherent attributes of graphene itself to reluctance to accept graphene in new manufacturing processes because of the uncertainty of the unknown. The 2D nature of graphene materials with a thickness of one atom presents significant challenges to proper handling/processing, and process scale-up becomes difficult because it requires high-end, expensive equipment, even for routine handling and analysis for quality assurance and control. Pristine graphene can also be extremely difficult to work into other matrices, thus hindering downstream processibility, especially at large scale. Currently, the cost of graphene and graphene products is still high and presents an economic risk that tends to slow down investment in scaling up emerging technologies. The lack of a standard for graphene materials for quality assurance and quality control poses a great challenge not only for the markets but also for commercialization efforts. A first-look economic feasibility analysis of the UCP technology provided valuable information that suggests the UCP process would be feasible, especially when it is scaled to a pilot scale and could be more competitive at the full scale. Graphitization was found to be the most energy-consuming and most capital-intensive step in the overall process. In small laboratory- and bench-scale experiments, labor is a significant contributor to the total process costs. Although these energy, capital, and labor constraints contribute to a higher selling price for the product, a preliminary economic model suggests that the process would be feasible at large scale when the process is fully integrated, optimized, and automated.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2021

Document Retrieval and Ranking using Similarity Graph Mean Hitting Times

We present a novel approach to information retrieval and document analysis based on graph analytic methods. Traditional information retrieval methods use a set of terms to define a query that is applied against a document corpus to identify the documents most related to those terms. In contrast, we define a query as a set of documents of interest and apply the query by computing mean hitting times between this set and all other documents on a document similarity graph abstraction of the semantic relationships between all pairs of documents. We present the steps of our approach along with a simple example application illustrating how this approach can be used to find documents related to two or more documents or topics of interest.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2025

Random Phase Approximation Correlation Energy Using Real-Space Density Functional Perturbation Theory

We present a real-space method for computing the random phase approximation (RPA) correlation energy within Kohn–Sham density functional theory, leveraging the low-rank nature of the frequency-dependent density response operator. In particular, we employ a cubic-scaling formalism based on density functional perturbation theory that circumvents the calculation of the response function matrix, instead relying on the ability to compute its product with a vector through the solution of the associated Sternheimer linear systems. We develop a large-scale parallel implementation of this formalism using the subspace iteration method in conjunction with the spectral quadrature method while employing the Kronecker product-based method for the application of the Coulomb operator and the conjugate orthogonal conjugate gradient method for the solution of the linear systems. We demonstrate convergence with respect to key parameters and verify the method’s accuracy by comparing with plane-wave results. We show that the framework achieves good strong scaling to many thousands of processors, reducing the time to solution for a lithium hydride system with 128 electrons to around 150 s on 4608 processors.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Applications of fuzzy logic and best-worst method for tritium sensor selection

Accurate assessment of tritium as a fuel source is critical in fusion reactions, necessitating effective sensor evaluation methods. This study investigates a multi-criteria decision-making framework for selecting tritium sensors, integrating fuzzy logic to enhance decision quality. Initial attempts at applying fuzzy logic were found to be too elementary and failed to capture the complexity of multi-criteria selection; this prompted a refined approach that incorporated expert insights and advanced ranking techniques for sensor evaluation. The research used a two-stage methodology. In the first stage, important criteria and sub-criteria for sensor performance were identified and defined. These criteria were then weighted and scored using a fuzzy best-worst method, drawing upon expert opinions to ensure relevance and validity. The second stage involved interpreting information about varying sensors to rank them based on their overall criteria scores, encouraging the selection of the most suitable options. The result of the study is a proposed method for effective sensor selection in fusion reactors, which in turn will significantly improve the reliability of tritium monitoring in fusion applications.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

134 billion intersecting brane models

The landscape of string vacua is very large, but generally expected to be finite in size. Enumerating the number and properties of the vacua is an important task for both the landscape and the swampland, in part to gain a deeper understanding of what is possible and “generic”. We obtain an exact counting of distinct intersecting brane vacua of type IIA string theory on the T 6 /Z 2 × Z 2 orientifold. Care is taken to only count gauge-inequivalent brane configurations. Leveraging the recursive nature by which branes may be added together one-by-one, we use dynamic programming to efficiently count the number of solutions of the tadpole, K-theory and supersymmetry consistency conditions. The distributions of 4D gauge group rank and complex structure moduli for the entire ensemble of intersecting brane vacua are presented. The methods we developed here may be useful in obtaining sharp upper and lower bounds on other corners of the landscape.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Scalable computations for nonstationary Gaussian processes

Nonstationary Gaussian process models can capture complex spatially varying dependence structures in spatial datasets. However, the large number of observations in modern datasets makes fitting such models computationally intractable with conventional dense linear algebra. In addition, derivative-free or even first-order optimization methods can be very slow to converge when estimating many spatially varying parameters. In this paper, we present a computational framework which couples an algebraic block diagonal plus low-rank covariance matrix approximation with stochastic trace estimation to facilitate the efficient use of second-order solvers for maximum likelihood estimation of Gaussian process models with many parameters. We demonstrate the effectiveness of these methods by simultaneously fitting 192 parameters in the popular nonstationary model of Paciorek and Schervish using 107,600 sea surface temperature anomaly measurements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗
DOE OSTI2022

DISTEMA: distance map-based estimation of single protein model accuracy with attentive 2D convolutional neural network

Abstract Background Estimation of the accuracy (quality) of protein structural models is important for both prediction and use of protein structural models. Deep learning methods have been used to integrate protein structure features to predict the quality of protein models. Inter-residue distances are key information for predicting protein’s tertiary structures and therefore have good potentials to predict the quality of protein structural models. However, few methods have been developed to fully take advantage of predicted inter-residue distance maps to estimate the accuracy of a single protein structural model. Result We developed an attentive 2D convolutional neural network (CNN) with channel-wise attention to take only a raw difference map between the inter-residue distance map calculated from a single protein model and the distance map predicted from the protein sequence as input to predict the quality of the model. The network comprises multiple convolutional layers, batch normalization layers, dense layers, and Squeeze-and-Excitation blocks with attention to automatically extract features relevant to protein model quality from the raw input without using any expert-curated features. We evaluated DISTEMA’s capability of selecting the best models for CASP13 targets in terms of ranking loss of GDT-TS score. The ranking loss of DISTEMA is 0.079, lower than several state-of-the-art single-model quality assessment methods. Conclusion This work demonstrates that using raw inter-residue distance information with deep learning can predict the quality of protein structural models reasonably well. DISTEMA is freely at https://github.com/jianlin-cheng/DISTEMA

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Generalized Canonical Polyadic Tensor Decomposition

Tensor decomposition is a fundamental unsupervised machine learning method in data science, with applications including network analysis and sensor data processing. This work develops a generalized canonical polyadic (GCP) low-rank tensor decomposition that allows other loss functions besides squared error. For instance, we can use logistic loss or Kullback--Leibler divergence, enabling tensor decomposition for binary or count data. We present a variety of statistically motivated loss functions for various scenarios. We provide a generalized framework for computing gradients and handling missing data that enables the use of standard optimization methods for fitting the model. Furthermore, we demonstrate the flexibility of the GCP decomposition on several real-world examples including interactions in a social network, neural activity in a mouse, and monthly rainfall measurements in India.

97 MATHEMATICS AND COMPUTING↗