Search NASA⌕ Search

Engineering topics

Wang, Bao

Publications and source records attributed to Wang, Bao.

Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

Abstract We prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.

Whiting, Wes (ORCID:0000000247505060)↗

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

Momentum is crucial in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, the momentum is usually weighted by a well-calibrated constant. However, tuning the hyperparameter for momentum can be a significant computational burden. In this article, we propose a novel adaptive momentum for improving DNNs training; this adaptive momentum, with no momentum-related hyperparame- ter required, is motivated by the nonlinear conjugate gradient (NCG) method. Stochastic gradient descent (SGD) with this new adaptive momentum eliminates the need for the momentum hyperparameter calibration, allows using a significantly larger learning rate, accelerates DNN training, and improves the final accuracy and robustness of the trained DNNs. For example, SGD with this adaptive momentum reduces classification errors for training ResNet110 for CIFAR10 and CIFAR100 from 5.25% to 4.64% and 23.75% to 20.03%, respectively. Furthermore, SGD, with the new adaptive momentum, also benefits adversarial training and, hence, improves the adversarial robustness of the trained DNNs.

97 MATHEMATICS AND COMPUTING↗

Accelerated Sparse Recovery via Gradient Descent with Nonlinear Conjugate Gradient Momentum

This paper applies an idea of adaptive momentum for the nonlinear conjugate gradient to accelerate optimization problems in sparse recovery. Specifically, we consider two types of minimization problems: a (single) differentiable function and the sum of a non-smooth function and a differentiable function. In the first case, we adopt a fixed step size to avoid the traditional line search and establish the convergence analysis of the proposed algorithm for a quadratic problem. This acceleration is further incorporated with an operator splitting technique to deal with the non-smooth function in the second case. As a result, we use the convex ι 1 and the nonconvex ι 1 – ι 2 functionals as two case studies to demonstrate the efficiency of the proposed approaches over traditional methods.

97 MATHEMATICS AND COMPUTING↗

Glassoformer: A Query-Sparse Transformer for Post-Fault Power Grid Voltage Prediction

Here, we propose GLassoformer, a novel and efficient transformer architecture leveraging group Lasso regularization to reduce the number of queries of the standard self-attention mechanism. Due to the sparsified queries, GLassoformer is more computationally efficient than the standard transformers. On the power grid post-fault voltage prediction task, GLasso-former shows remarkably better prediction than many existing benchmark algorithms in terms of accuracy and stability.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo

As an important Markov chain Monte Carlo (MCMC) method, the stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling. Furthermore, SGLD typically suffers from a slow convergence rate due to its large variance caused by the stochastic gradient. In order to alleviate these drawbacks, we leverage the recently developed Laplacian smoothing technique and propose a Laplacian smoothing stochastic gradient Langevin dynamics (LS-SGLD) algorithm. We prove that for sampling from both log-concave and non-log-concave densities, LS-SGLD achieves strictly smaller discretization error in 2-Wasserstein distance, although its mixing rate can be slightly slower. Experiments on both synthetic and real datasets verify our theoretical results and demonstrate the superior performance of LS-SGLD on different machine learning tasks including posterior sampling, Bayesian logistic regression, and training Bayesian convolutional neural networks.

97 MATHEMATICS AND COMPUTING↗

Consolidating Lithiothermic-Ready Transition Metals for Li2S-Based Cathodes

Li 2 S holds a promising role as a high-capacity Li-containing cathode, circumventing use of metallic lithium in constructing next-generation batteries to replace current Li-ion batteries. However, progress of Li 2 S cathode has been plagued by its intrinsic drawbacks, including high activation potentials, poor rate performance, and rapid capacity fading during long cycling. In this work, a series of Li 2 S/transition metal (TM) nanocomposites are synthesized via a lithiothermic reduction reaction, and it is realized that the presence of TMs in Li 2 S matrix can transform electrochemical behaviors of Li 2 S. On the one hand, the incorporation of W, Mo, or Ti greatly increases electronic and ionic conductivity of Li 2 S composites and inhibits the polysulfide dissolution via the TM—S bond, effectively addressing the drawbacks of Li 2 S cathodes. In particular, Li 2 S/W and Li 2 S/Mo exhibit the highest ionic conductivity of solid-phase Li-ion conductors ever-reported: 5.44 × 10 –2 and 3.62 × 10 –2 S m –1 , respectively. On the other hand, integrating Co, Mn, and Zn turns Li 2 S into a prelithiation agent, forming metal sulfides rather than S 8 after the full charge. These interesting findings may shed light on the design of Li 2 S-based cathode materials.

25 ENERGY STORAGE↗

Formation of Plastic Inclusions in U71Mnk High-Speed Heavy-Rail Steel Refined by CaO-SiO{sub 2}-Al{sub 2}O{sub 3}-MgO Slag

To precisely control the characteristics of nonmetallic inclusions with high plasticity in U71Mnk high-speed heavy-rail steel deoxidized by Si and Mn, the chemical composition of CaO-SiO{sub 2}-Al{sub 2}O{sub 3}-MgO slag in the ladle furnace (LF) refining process was optimized by thermodynamic calculations. Relationships among the refining slag, nonmetallic inclusions, and elemental concentrations in the steels were analyzed to obtain the conditions for determining the optimal composition of refining slag. The optimal compositions and the component activities of CaO-SiO{sub 2}-Al{sub 2}O{sub 3}-MgO refining slag were determined and discussed according to calculations using FactSage 7.0. Finally, ten industrial experiments were performed. Variations in the compositions of LF refining slag, inclusions, and steel, as well as the sizes of the inclusions, and metallographic classification assessment of the U71Mnk steel product, were investigated and discussed to verify the feasibility and effectiveness of the optimized LF refining slag. Kinetics of steel–slag and steel-inclusion reactions during LF refining was studied and compared. The results show that to obtain low [O], [Al], and [S] contents and inclusions with high plasticity in molten steel, the optimal composition of the LF refining slag should be CaO: 44.2 to 51.7 wt pct, SiO{sub 2}: 41.4 to 47.0 wt pct, Al{sub 2}O{sub 3}: 0 to 4.7 wt pct and MgO 5.0 to 7.0 wt pct, where the basicity R is above 1.00 and the C/A ratio is above 9.00. After the LF refining process, the contents of total oxygen (T.O.), [Al], and [S] gradually stabilized in the range of 0.0008 to 0.0012 wt pct, 0.002 to 0.0028 wt pct and 0.0026 to 0.0037 wt pct, respectively. The densities of the three kinds of typical inclusions (CaO-SiO{sub 2}-Al{sub 2}O{sub 3}-MgO, MnS, and CaO-SiO{sub 2}-Al{sub 2}O{sub 3}-MgO-MnS) also decreased and reached approximately 0.25/mm{sup 2}, and no inclusion larger than 6.0 μm was found. The composition of the inclusions gradually changed into the optimal range in the CaO-SiO{sub 2}-Al{sub 2}O{sub 3}-MgO slag system during the LF refining process.

36 MATERIALS SCIENCE↗

Scientific data interpolation with low dimensional manifold model

Here, we propose to apply a low dimensional manifold model to scientific data interpolation from regular and irregular samplings with a significant amount of missing information. The low dimensionality of the patch manifold for general scientific data sets has been used as a regularizer in a variational formulation. The problem is solved via alternating minimization with respect to the manifold and the data set, and the Laplace–Beltrami operator in the Euler–Lagrange equation is discretized using the weighted graph Laplacian. Various scientific data sets from different fields of study are used to illustrate the performance of the proposed algorithm on data compression and interpolation from both regular and irregular samplings.

97 MATHEMATICS AND COMPUTING↗

Energy storage device including a redox-enhanced electrolyte

An electrical double layer capacitor (EDLC) energy storage device is provided that includes at least two electrodes and a redox-enhanced electrolyte including two redox couples such that there is a different one of the redox couples for each of the electrodes. When charged, the charge is stored in Faradaic reactions with the at least two redox couples in the electrolyte and in a double-layer capacitance of a porous carbon material that comprises at least one of the electrodes, and a self-discharge of the energy storage device is mitigated by at least one of electrostatic attraction, adsorption, physisorption, and chemisorption of a redox couple onto the porous carbon material.

25 ENERGY STORAGE↗

Dynamics and heterogeneity of a fate determinant during transition towards cell differentiation

Yan is an ETS-domain transcription factor responsible for maintaining Drosophila eye cells in a multipotent state. Yan is at the core of a regulatory network that determines the time and place in which cells transit from multipotency to one of several differentiated lineages. Using a fluorescent reporter for Yan expression, we observed a biphasic distribution of Yan in multipotent cells, with a rapid inductive phase and slow decay phase. Transitions to various differentiated states occurred over the course of this dynamic process, suggesting that Yan expression level does not strongly determine cell potential. Consistent with this conclusion, perturbing Yan expression by varying gene dosage had no effect on cell fate transitions. However, we observed that as cells transited to differentiation, Yan expression became highly heterogeneous and this heterogeneity was transient. Signals received via the EGF Receptor were necessary for the transience in Yan noise since genetic loss caused sustained noise. Since these signals are essential for eye cells to differentiate, we suggest that dynamic heterogeneity of Yan is a necessary element of the transition process, and cell states are stabilized through noise reduction.

59 BASIC BIOLOGICAL SCIENCES↗