Search NASA⌕ Search

Engineering topics

Vassilevski, Panayot

Publications and source records attributed to Vassilevski, Panayot.

Multilevel-in-width training for deep neural network regression

A common challenge in regression is that for many problems, the degrees of freedom required for a high-quality solution also allows for overfitting. Regularization is a class of strategies that seek to restrict the range of possible solutions so as to discourage overfitting while still enabling good solutions, and different regularization strategies impose different types of restrictions. In this paper, we present a multilevel regularization strategy that constructs and trains a hierarchy of neural networks, each of which has layers that are wider versions of the previous network's layers. Here, we draw intuition and techniques from the field of Algebraic Multigrid (AMG), traditionally used for solving linear and nonlinear systems of equations, and specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of equations to the problem of deep learning. Training through V-cycles then encourage the neural networks to build a hierarchical understanding of the problem. We refer to this approach as multilevel-in-width to distinguish from prior multilevel works which hierarchically alter the depth of neural networks. The resulting approach is a highly flexible framework that can be applied to a variety of layer types, which we demonstrate with both fully connected and convolutional layers. We experimentally show with PDE regression problems that our multilevel training approach is an effective regularizer, improving the generalize performance of the neural networks studied.

97 MATHEMATICS AND COMPUTING↗

A Condensed Constrained Nonconforming Mortar-Based Approach for Preconditioning Finite Element Discretization Problems

This paper presents and studies an approach for constructing auxiliary space preconditioners for finite element problems using a constrained nonconforming reformulation that is based on a proposed modified version of the mortar method. The well-known mortar finite element discretization method is modified to admit a local structure, providing an element-by-element or subdomain-by-subdomain assembly property. This is achieved via the introduction of additional trace finite element spaces and degrees of freedom (unknowns) associated with the interfaces between adjacent elements or subdomains. The resulting nonconforming formulation and a reduced-via-static-condensation Schur complement form on the interfaces are used in the construction of auxiliary space preconditioners for a given conforming finite element discretization problem. Overall, the properties of these preconditioners are studied and their performance is illustrated on model second order scalar elliptic problems utilizing high order elements.

97 MATHEMATICS AND COMPUTING↗

Comparative Performance Analysis of Coarse Solvers for Algebraic Multigrid on Multicore and Manycore Architectures

In this paper, we study the performance of a two-level algebraic-multigrid algorithm, with a focus on the impact of the coarse-grid solver on performance. We consider two algorithms for solving the coarse-space systems: the preconditioned conjugate gradient method and a new robust HSS-embedded low-rank sparse-factorization algorithm. Our test data comes from the SPE Comparative Solution Project for oil-reservoir simulations. We contrast the performance of our code on one 12-core socket of a Cray XC30 machine with performance on a 60-core Intel Xeon Phi coprocessor. To obtain top performance, we optimized the code to take full advantage of fine-grained parallelism and made it thread-friendly for high thread count. We also developed a bounds-and-bottlenecks performance model of the solver which we used to guide us through the optimization effort, and also carried out performance tuning in the solver’s large parameter space. Finally, as a result, significant speedups were obtained on both machines.

97 MATHEMATICS AND COMPUTING↗

Bayesian Uncertainty Quantification in Predictions of Flows in Highly Heterogeneous Media and Its Applications to the CO 2 Sequestration

In this proposal, we have worked on Bayesian uncertainty quantification for predictions of fows in highly heterogeneous media. The research in this proposal is broad and includes: prior modeling for heterogeneous permeability fields; effective parametrization of heterogeneous spatial priors; efficient ensemble- level solution techniques; efficient multiscale approximation techniques; study of the regularity of complex posterior distribution and the error estimates due to parameter reduction; efficient sampling techniques; applications to multi-phase ow and transport. We list our publications below and describe some of our main research activities. Our multi-disciplinary team includes experts from the areas of multiscale modeling, multilevel solvers, Bayesian statistics, spatial permeability modeling, and the application domain.

97 MATHEMATICS AND COMPUTING↗

A Novel Vertex Affinity for Community Detection

We propose a novel vertex affinity measure in this paper. The new vertex affinity quantifies the proximity between two vertices in terms of their clustering strength and is ideal for such graph analytics applications as community detection. We also developed a framework that combines simple graph searches and resistance circuit formulas to compute the vertex affinity efficiently. We study the properties of the new affinity measure empirically in comparison to those of other popular vertex proximity metrics. Our results show that the existing metrics are ill-suited for community detection due to their lack of fundamental properties that are essential for correctly capturing inter- and intra-cluster vertex proximity.

97 MATHEMATICS AND COMPUTING↗

Enhancing Community Detection By Affinity-based Edge Weighting Scheme

Community detection refers to an important graph analytics problem of finding a set of densely-connected subgraphs in a graph and has gained a great deal of interest recently. The performance of current community detection algorithms is limited by an inherent constraint of unweighted graphs that offer very little information on their internal community structures. In this paper, we propose a new scheme to address this issue that weights the edges in a given graph based on recently proposed vertex affinity. The vertex affinity quantifies the proximity between two vertices in terms of their clustering strength, and therefore, it is ideal for graph analytics applications such as community detection. We also demonstrate that the affinity-based edge weighting scheme can improve the performance of community detection algorithms significantly.

97 MATHEMATICS AND COMPUTING↗

Regular Decompositions for H(div) Spaces

We study regular decompositions for H(div) spaces. In particular, we show that such regular decompositions are closely related to a previously studied “inf-sup” condition for parameter-dependent Stokes problems, for which we provide an alternative, more direct, proof.

97 MATHEMATICS AND COMPUTING↗