Search NASA⌕ Search

SEARCH · Search NASA

Results for “COMPUTING PROBLEMS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Parallel Multivariate Spatio-Temporal Clustering of Large Ecological Datasets on Hybrid Supercomputers

A proliferation of data from vast networks of remote sensing platforms (satellites, unmanned aircraft systems (UAS), airborne etc.), observational facilities (meteorological, eddy covariance etc.), state-of-the-art sensors, and simulation models offer unprecedented opportunities for scientific discovery. Unsupervised classification is a widely applied data mining approach to derive insights from such data. However, classification of very large data sets is a complex computational problem that requires efficient numerical algorithms and implementations on high performance computing (HPC) platforms. Additionally, increasing power, space, cooling and efficiency requirements has led to the deployment of hybrid supercomputing platforms with complex architectures and memory hierarchies like the Titan system at Oak Ridge National Laboratory. The advent of such accelerated computing architectures offers new challenges and opportunities for big data analytics in general and specifically, large scale cluster analysis in our case. Although there is an existing body of work on parallel cluster analysis, those approaches do not fully meet the needs imposed by the nature and size of our large data sets. Moreover, they had scaling limitations and were mostly limited to traditional distributed memory computing platforms. We present a parallel Multivariate Spatio-Temporal Clustering (MSTC) technique based on k-means cluster analysis that can target hybrid supercomputers like Titan. We developed a hybrid MPI, CUDA and OpenACC implementation that can utilize both CPU and GPU resources on computational nodes. We describe performance results on Titan that demonstrate the scalability and efficacy of our approach in processing large ecological data sets.

Sreepathi, Sarat↗

Frog: Asynchronous Graph Processing on GPU with Hybrid Coloring Model

GPUs have been increasingly used to accelerate graph processing for complicated computational problems regarding graph theory. Many parallel graph algorithms adopt the asynchronous computing model to accelerate the iterative convergence. Unfortunately, the consistent asynchronous computing requires locking or atomic operations, leading to significant penalties/overheads when implemented on GPUs. As such, coloring algorithm is adopted to separate the vertices with potential updating conflicts, guaranteeing the consistency/correctness of the parallel processing. Common coloring algorithms, however, may suffer from low parallelism because of a large number of colors generally required for processing a large-scale graph with billions of vertices. We propose a light-weight asynchronous processing framework called Frog with a preprocessing/hybrid coloring model. The fundamental idea is based on Pareto principle (or 80-20 rule) about coloring algorithms as we observed through masses of realworld graph coloring cases. We find that a majority of vertices (about 80%) are colored with only a few colors, such that they can be read and updated in a very high degree of parallelism without violating the sequential consistency. Accordingly, our solution separates the processing of the vertices based on the distribution of colors. In this work, we mainly answer three questions: (1) how to partition the vertices in a sparse graph with maximized parallelism, (2) how to process large-scale graphs that cannot fit into GPU memory, and (3) how to reduce the overhead of data transfers on PCIe while processing each partition. Here, we conduct experiments on real-world data (Amazon, DBLP, YouTube, RoadNet-CA, WikiTalk and Twitter) to evaluate our approach and make comparisons with well-known non-preprocessed (such as Totem, Medusa, MapGraph and Gunrock) and preprocessed (Cusha) approaches, by testing four classical algorithms (BFS, PageRank, SSSP and CC). On all the tested applications and datasets, Frog is able to significantly outperform existing GPU-based graph processing systems except Gunrock and MapGraph. MapGraph gets better performance than Frog when running BFS on RoadNet-CA. The comparison between Gunrock and Frog is inconclusive. Frog can outperform Gunrock more than 1.04X when running PageRank and SSSP, while the advantage of Frog is not obvious when running BFS and CC on some datasets especially for RoadNet-CA.

97 MATHEMATICS AND COMPUTING↗

Fast simultaneous feasibility testing and applications for electrical power grids

Fast simultaneous feasibility testing (SFT) for management of an electrical power grid is achieved through various innovations. The computation problem relates to evaluation of candidate solutions for external power flows into a power grid, with respect to predetermined constraints and contingencies. Storage and computations are reduced by formulating the problem in terms of transactional nodes (e.g. third party connections for generators and loads) instead of the larger number of bus nodes. Further advantages are achieved by precomputing matrices that can be reused across multiple SFT invocations, organizing matrices and operations to reduce storage and computation, and eliminating branches that have no contingency violations. The fast SFT further enables new applications for integrating SFT with a Security-Constrained Unit Commitment (SCUC) optimizer for efficient single pass solutions; operating SFT as a service to multiple SCUC solvers; or integration with Security-Constrained Economic Dispatch (SCED) optimization.

97 MATHEMATICS AND COMPUTING↗

Multiscale/Multiphysics Modeling of Biomass Thermochemical Processes

Computational problems in simulating biomass thermochemical processes involve coupling processes that span several orders of magnitude in space and time. Computational difficulties arise from the multitude of the problem governing equations, each typically applying over a narrow range of spatiotemporal scales, thus making it necessary to represent the processes as the result of the interaction of multiple physics modules, termed here as multiscale/multiphysics (MSMP) coupling. Predictive simulations for such processes require algorithms that can efficiently integrate the underlying MSMP methods across the scales in order to achieve prescribed accuracy and control the computational cost. In addition, MSMP algorithms must scale to one hundred thousand processors or more in order to effectively harness the new computational resources and accelerate the scientific advances. In this chapter, we discuss the state-of-the-art in modeling the macro-scale phenomena in a biomass pyrolysis reactor along with details of the shortcomings and prospects in improving predictability. We also introduce the various multiphysics modules needed to model thermochemical conversion at lower spatiotemporal scales. Furthermore, we illustrate the need for MSMP coupling for thermochemical processes in biomass and provide an overview of the wavelet-based coupling techniques we have developed recently. In particular, we provide details about the compound wavelet matrix (CWM) and the dynamic CWM (dCWM) methods and show they are highly efficient in transferring information among multiphysics models across multiple temporal and spatial scales. The algorithmic gain is in addition to the parallel spatial scalability from traditional domain decomposition methods. The CWM algorithms are serial in time and limited by the smallest-system time-scales. In order to relax this algorithmic constraint, we have recently coupled time parallel (TP) algorithms to CWM, thus yielding a novel approach termed tpCWM. We present preliminary results from the tpCWM technique, indicating that we can accelerate time-to-solution by 2 to 3-orders of magnitude even on 20-processors and this can potentially constitute a new paradigm for MSMP simulations. If such improvements in simulation capability can be generalized, the tpCWM approach can lead the way to predictive simulations of biomass thermochemical processes.

09 BIOMASS FUELS↗

Post-processing interstitialcy diffusion from molecular dynamics simulations

An algorithm to rigorously trace the interstitialcy diffusion trajectory in crystals is developed. The algorithm incorporates unsupervised learning and graph optimization which obviate the need to input extra domain specific information depending on crystal or temperature of the simulation. The algorithm is implemented in a flexible framework as a post-processor to molecular dynamics (MD) simulations. We describe in detail the reduction of interstitialcy diffusion into known computational problems of unsupervised clustering and graph optimization. We also discuss the steps, computational efficiency and key components of the algorithm. Using the algorithm, thermal interstitialcy diffusion from low to near-melting point temperatures is studied. We encapsulate the algorithms in a modular framework with functionality to calculate diffusion coefficients, migration energies and other trajectory properties. The study validates the algorithm by establishing the conformity of output parameters with experimental values and provides detailed insights for the interstitialcy diffusion mechanism. The algorithm along with the help of supporting visualizations and analysis gives convincing details and a new approach to quantifying diffusion jumps, jump-lengths, time between jumps and to identify interstitials from lattice atoms. -- Graphical abstract:.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Surrogate-Based Autotuning for Randomized Sketching Algorithms in Regression Problems

Algorithms from Randomized Numerical Linear Algebra (RandNLA) are known to be effective in handling high-dimensional computational problems, providing high-quality empirical performance as well as strong probabilistic guarantees. However, their practical application is complicated by the fact that the user needs to set various algorithm-specific tuning parameters which are different from those used in traditional NLA. This paper demonstrates how a surrogate-based autotuning approach can be used to address fundamental problems of parameter selection in RandNLA algorithms. In particular, we provide a detailed investigation of surrogate-based autotuning for sketch-and-precondition (SAP)-based randomized least squares methods, which have been one of the great success stories in modern RandNLA. Empirical results show that our surrogate-based autotuning approach can achieve near-optimal performance with much less tuning cost than a random search (up to about 7.6x fewer trials of different parameter configurations). Moreover, while our experiments focus on least squares, our results demonstrate a general-purpose autotuning pipeline applicable to any kind of RandNLA algorithm.

Cho, Younghyun↗

In Defense of the National Labs and Big-Budget Science

The purpose of this paper is to present the unofficial and unsanctioned opinions of a Visiting Scientist at Lawrence Livermore National Laboratory on the values of LLNL and the other National Labs. The basic founding value and goal of the National Labs is big-budget scientific research, along with smaller-budget scientific research that cannot easily be done elsewhere. The most important example in the latter category is classified defense-related research. The historical guiding light here is the Manhattan Project. This endeavor was unique in human history, and might remain so. The scientific expertise and wealth of an entire nation was tapped in a project that was huge beyond reckoning, with no advance guarantee of success. It was in many respects a clash of scientific titans, with a large supporting cast, collaborating toward a single well-defined goal. Never had scientists received so much respect, so much money, and so much intellectual freedom to pursue scientific progress. And never was the gap between theory and implementation so rapidly narrowed, with results that changed the world, completely. Enormous resources are spent at the national or international level on large-scale scientific projects. LLNL has the most powerful computer in the world, Blue Gene/L. (Oops, Los Alamos just seized the title with Roadrunner; such titles regularly change hands.) LLNL also has the largest laser in the world, the National Ignition Facility (NIF). Lawrence Berkeley National Lab (LBNL) has the most powerful microscope in the world. Not only is it beyond the resources of most large corporations to make such expenditures, but the risk exceeds the possible rewards for those corporations that could. Nor can most small countries afford to finance large scientific projects, and not even the richest can afford largess, especially if Congress is under major budget pressure. Some big-budget research efforts are funded by international consortiums, such as the Large Hadron Collider (LHC) at CERN, and the International Tokamak Experimental Reactor (ITER) in Cadarache, France, a magnetic-confinement fusion research project. The postWWII histories of particle and fusion physics contain remarkable examples of both international competition, with an emphasis on secrecy, and international cooperation, with an emphasis on shared knowledge and resources. Initiatives to share sometimes came from surprising directions. Most large-scale scientific projects have potential defense applications. NIF certainly does; it is primarily designed to create small-scale fusion explosions. Blue Gene/L operates in part in service to NIF, and in part to various defense projects. The most important defense projects include stewardship of the national nuclear weapons stockpile, and the proposed redesign and replacement of those weapons with fewer, safer, more reliable, longer-lived, and less apocalyptic warheads. Many well-meaning people will consider the optimal lifetime of a nuclear weapon to be zero, but most thoughtful people, when asked how much longer they think this nation will require them, will ask for some time to think. NIF is also designed to create exothermic small-scale fusion explosions. The malapropos 'exothermic' here is a convenience to cover a profusion of complexities, but the basic idea is that the explosions will create more recoverable energy than was used to create them. One can hope that the primary future benefits of success for NIF will be in cost-effective generation of electrical power through controlled small-scale fusion reactions, rather than in improved large-scale fusion explosions. Blue Gene/L also services climate research, genomic research, materials research, and a myriad of other computational problems that become more feasible, reliable, and precise the larger the number of computational nodes employed. Blue Gene/L has to be sited within a security complex for obvious reasons, but its value extends to the nation and the world. There is a duality here between large-scale scientific research machines and the supercomputers used to model them. An astounding example is illustrated in a graph released by EFDAJET, at Oxfordshire, UK, presently the largest operating magnetic-confinement fusion experiment. The graph shows plasma confinement times (an essential performance parameter) for all the major tokamaks in the international fusion program, over their existing lifetimes. The remarkable thing about the data is not so much confinement-time versus date or scale, but the fact that the data are given for both the computer model predictions and the actual experimental measurements, and the two are in phenomenal agreement over the extended range of scales. Supercomputer models, sometimes operating with the intricacy of Schroedinger's equation at quantum physical scales, have become a costly but enormously cost-saving tool.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Constructing the Pareto front for multi-objective Markov chains handling a strong Pareto policy approach

In this paper, we present a multi-objective solution of the Pareto front for a certain class of discrete-time ergodic controllable Markov chains. For solving the problem, we transform the original multi-objective problem into an equivalent nonlinear programming problem implementing the Lagrange principle. We then propose to adopt the Tikhonov’s regularization method to ensure the convergence of the cost functions to a unique point into the Pareto front. We show that Pareto policies are characterized as optimal policies. One of the fundamental problems related to the construction of the Pareto front is the existence and characterization of both, Pareto and strong Pareto policies. By introducing the Tikhonov’s regularizator, we ensure the existence of strong Pareto policies. This paper proposes a real solution to this problem. We formulate the original problem considering several constraints: (a) we employ the c-variable method for the introduction of linear constraints over the nonlinear problem and (b) restrict the cost functions, allowing points in the Pareto front to have a small distance from one another. The constraints imposed by the c-variable method make the problem computationally tractable and the restriction imposed by the small distance change ensures the continuation of the Pareto front. The resulting equation in this nonlinear system is an optimization problem for which the necessary and efficient condition of a minimum is solved using the projected gradient method. Moreover, we also prove the convergence conditions and compute the estimate rate of convergence of variables corresponding to the Lagrange principle and the Tikhonov’s regularization, respectively. We provide all the details needed to implement the proposed method in an efficient way. The usefulness of the method is successfully demonstrated by a numerical example.

97 MATHEMATICS AND COMPUTING↗

Electromagnetic Simulation of the Near-Field Distribution around a Wind Farm

An efficient approach to compute the near-field distribution around and within a wind farm under plane wave excitation is proposed. To make the problem computationally tractable, several simplifying assumptions are made based on the geometry problem. By comparing the approximations against full-wave simulations at 500 MHz, it is shown that the assumptions do not introduce significant errors into the resulting near-field distribution. The near fields around a 3 × 3 wind farm are computed using the developed methodology at 150 MHz, 500 MHz, and 3 GHz. Both the multipath interference patterns and the forward shadows are predicted by the proposed method.

Yang, Shang-Te↗

Hierarchical off-diagonal low-rank approximation of Hessians in inverse problems, with application to ice sheet model initialization

Obtaining lightweight and accurate approximations of discretized objective functional Hessians in inverse problems governed by partial differential equations (PDEs) is essential to make both deterministic and Bayesian statistical large-scale inverse problems computationally tractable. The cubic computational complexity of dense linear algebraic tasks, such as Cholesky factorization, that provide a means to sample Gaussian distributions and determine solutions of Newton linear systems is a computational bottleneck at large-scale. These tasks can be reduced to log-linear complexity by utilizing hierarchical off-diagonal low-rank (HODLR) matrix approximations. In this work, we show that a class of Hessians that arise from inverse problems governed by PDEs are well approximated by the HODLR matrix format. In particular, we study inverse problems governed by PDEs that model the instantaneous viscous flow of ice sheets. In these problems, we seek a spatially distributed basal sliding parameter field such that the flow predicted by the ice sheet model is consistent with ice sheet surface velocity observations. Here, we demonstrate the use of HODLR Hessian approximation to efficiently sample the Laplace approximation of the posterior distribution with covariance further approximated by HODLR matrix compression. Computational studies are performed which illustrate ice sheet problem regimes for which the Gauss–Newton data-misfit Hessian is more efficiently approximated by the HODLR matrix format than the low-rank (LR) format. We then demonstrate that HODLR approximations can be favorable, when compared to global LR approximations, for large-scale problems by studying the data-misfit Hessian associated with inverse problems governed by the first-order Stokes flow model on the Humboldt glacier and Greenland ice sheet.

97 MATHEMATICS AND COMPUTING↗

Software Systems for High-performance Quantum Computing

Quantum computing promises new opportunities for solving hard computational problems, but harnessing this novelty requires breakthrough concepts in the design, operation, and application of computing systems. We define some of the challenges facing the development of quantum computing systems as well as software-based approaches that can be used to overcome these challenges. Following a brief overview of the state of the art, we present models for the quantum programming and execution models, the development of architectures for hybrid high-performance computing systems, and the realization of software stacks for quantum networking. This leads to a discussion of the role that conventional computing plays in the quantum paradigm and how some of the current challenges for exascale computing overlap with those facing quantum computing.

high-performance computing↗

Two-particle quantum walks applied to the graph isomorphism problem

We show that the quantum dynamics of interacting and noninteracting quantum particles are fundamentally different in the context of solving a particular computational problem. Specifically, we consider the graph isomorphism problem, in which one wishes to determine whether two graphs are isomorphic (related to each other by a relabeling of the graph vertices), and focus on a class of graphs with particularly high symmetry called strongly regular graphs (SRGs). We study the Green's functions that characterize the dynamical evolution single-particle and two-particle quantum walks on pairs of nonisomorphic SRGs and show that interacting particles can distinguish nonisomorphic graphs that noninteracting particles cannot. We obtain the following specific results. (1) We prove that quantum walks of two noninteracting particles, fermions or bosons, cannot distinguish certain pairs of nonisomorphic SRGs. (2) We demonstrate numerically that two interacting bosons are more powerful than single particles and two noninteracting particles, in that quantum walks of interacting bosons distinguish all nonisomorphic pairs of SRGs that we examined. By utilizing high-throughput computing to perform over 500 million direct comparisons between evolution operators, we checked all tabulated pairs of nonisomorphic SRGs, including graphs with up to 64 vertices. (3) By performing a short-time expansion of the evolution operator, we derive distinguishing operators that provide analytic insight into the power of the interacting two-particle quantum walk.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A review of non-cognitive applications for neuromorphic computing

Abstract Though neuromorphic computers have typically targeted applications in machine learning and neuroscience (‘cognitive’ applications), they have many computational characteristics that are attractive for a wide variety of computational problems. In this work, we review the current state-of-the-art for non-cognitive applications on neuromorphic computers, including simple computational kernels for composition, graph algorithms, constrained optimization, and signal processing. We discuss the advantages of using neuromorphic computers for these different applications, as well as the challenges that still remain. The ultimate goal of this work is to bring awareness to this class of problems for neuromorphic systems to the broader community, particularly to encourage further work in this area and to make sure that these applications are considered in the design of future neuromorphic systems.

97 MATHEMATICS AND COMPUTING↗

Computational Needs for the Next Generation Electric Grid Proceedings

The April 2011 DOE workshop, 'Computational Needs for the Next Generation Electric Grid', was the culmination of a year-long process to bring together some of the Nation's leading researchers and experts to identify computational challenges associated with the operation and planning of the electric power system. The attached papers provide a journey into these experts' insights, highlighting a class of mathematical and computational problems relevant for potential power systems research. While each paper defines a specific problem area, there were several recurrent themes. First, the breadth and depth of power system data has expanded tremendously over the past decade. This provides the potential for new control approaches and operator tools that can enhance system efficiencies and improve reliability. However, the large volume of data poses its own challenges, and could benefit from application of advances in computer networking and architecture, as well as data base structures. Second, the computational complexity of the underlying system problems is growing. Transmitting electricity from clean, domestic energy resources in remote regions to urban consumers, for example, requires broader, regional planning over multi-decade time horizons. Yet, it may also mean operational focus on local solutions and shorter timescales, as reactive power and system dynamics (including fast switching and controls) play an increasingly critical role in achieving stability and ultimately reliability. The expected growth in reliance on variable renewable sources of electricity generation places an exclamation point on both of these observations, and highlights the need for new focus in areas such as stochastic optimization to accommodate the increased uncertainty that is occurring in both planning and operations. Application of research advances in algorithms (especially related to optimization techniques and uncertainty quantification) could accelerate power system software tool performance, i.e. speed to solution, and enhance applicability for new and existing real-time operation and control approaches, as well as large-scale planning analysis. Finally, models are becoming increasingly essential for improved decision-making across the electric system, from resource forecasting to adaptive real-time controls to online dynamics analysis. The importance of data is thus reinforced by their inescapable role in validating, high-fidelity models that lead to deeper system understanding. Traditional boundaries (reflecting geographic, institutional, and market differences) are becoming blurred, and thus, it is increasingly important to address these seams in model formulation and utilization to ensure accuracy in the results and achieve predictability necessary for reliable operations. Each paper also embodies the philosophy that our energy challenges require interdisciplinary solutions - drawing on the latest developments in fields such as mathematics, computation, economics, as well as power systems. In this vein, the workshop should be viewed not as the end product, but the beginning of what DOE seeks to establish as a vibrant, on-going dialogue among these various communities. Bridging communication gaps among these communities will yield opportunities for innovation and advancement. The papers and workshop discussion provide the opportunity to learn from experts on the current state-of-the-art on computational approaches for electric power systems, and where one may focus to accelerate progress. It has been extremely valuable to me as I better understand this space, and consider future programmatic activities. I am confident that you too will enjoy the discussion, and certainly learn from the many experts. I would like to thank the authors of the papers for sharing their perspectives, as well as the paper discussants, session recorders, and participants. The meeting would not have been as successful without your commitment and engagement. I also would like to thank Joe Eto and Bob Thomas for their vision and leadership in bringing together such a well-structured and productive forum.

24 POWER TRANSMISSION AND DISTRIBUTION↗

GENERALIZATION OF THE LAST SCATTERING APPROXIMATION FOR THE SECOND SOLAR SPECTRUM MODELING: THE Ca I 4227 A LINE AS A CASE STUDY

To model the second solar spectrum (the linearly polarized spectrum of the Sun that is due to coherent scattering processes), one needs to solve the polarized radiative transfer (RT) equation. For strong resonance lines, partial frequency redistribution (PRD) effects must be accounted for, which make the problem computationally demanding. The 'last scattering approximation' (LSA) is a concept that has been introduced to make this highly complex problem more tractable. An earlier application of a simple LSA version could successfully model the wings of the strong Ca I 4227 A resonance line in Stokes Q/I (fractional linear polarization), but completely failed to reproduce the observed Q/I peak in the line core. Since the magnetic field signatures from the Hanle effect only occur in the line core, we need to generalize the existing LSA approach if it is to be useful for the diagnostics of chromospheric and turbulent magnetic fields. In this paper, we explore three different approximation levels for LSA and compare each of them with the benchmark represented by the solution of the full polarized RT, including PRD effects. The simplest approximation level is LSA-1, which uses the observed center-to-limb variation of the intensity profile to obtain the anisotropy of the radiation field at the surface, without solving any transfer equation. In contrast, the next two approximation levels use the solution of the unpolarized transfer equation to derive the anisotropy of the incident radiation field and use it as an input. In the case of LSA-2, the anisotropy at level {tau}{sub {lambda}} = {mu}, the atmospheric level from which an observed photon is most likely to originate, is used. LSA-3, on the other hand, makes use of the full depth dependence of the radiation anisotropy. The Q/I formula for LSA-3 is obtained by keeping the first term in a series expansion of the Q-source function in powers of the mean number of scattering events. Computationally, LSA-1 is 21 times faster than LSA-2, which is 5 times faster than the more general LSA-3, which itself is 8 times faster than the polarized RT approach. A comparison of the calculated Q/I spectra with the RT benchmark shows excellent agreement for LSA-3, including good modeling of the Q/I core region with its PRD effects. In contrast, both LSA-1 and LSA-2 fail to model the core region. The RT and LSA-3 approaches are then applied to model the recently observed Q/I profile of the Ca I 4227 A line in quiet regions of the Sun. Apart from a global scale factor both give a very good fit to the Q/I spectra for all the wavelengths, including the core peak and blend line depolarizations. We conclude that LSA-3 is an excellent substitute for the full polarized RT and can be used to interpret the second solar spectrum, including the Hanle effect with PRD. It also allows the techniques developed for unpolarized three-dimensional RT to be applied to the modeling of the second solar spectrum.

79 ASTRONOMY AND ASTROPHYSICS↗

Numerical simulation projects in micromagnetics with Jupyter

We report a case study where an existing materials science course was modified to include numerical simulation projects on the micromagnetic behavior of materials. The Ubermag micromagnetic simulation software package is used in order to solve problems computationally. The simulation software is controlled through the Python code in Jupyter notebooks. Our experience is that the self-paced problem-solving nature of the project work can facilitate a better in-depth exploration of the course contents. We discuss which aspects of the Ubermag and the project Jupyter ecosystem have been beneficial for the students' learning experience and which could be transferred to similar teaching activities in other subject areas.

97 MATHEMATICS AND COMPUTING↗

NVMalloc: Exposing an Aggregate SSD Store as a Memory Partition in Extreme-Scale Machines

DRAM is a precious resource in extreme-scale machines and is increasingly becoming scarce, mainly due to the growing number of cores per node. On future multi-petaflop and exaflop machines, the memory pressure is likely to be so severe that we need to rethink our memory usage models. Fortunately, the advent of non-volatile memory (NVM) offers a unique opportunity in this space. Current NVM offerings possess several desirable properties, such as low cost and power efficiency, but suffer from high latency and lifetime issues. We need rich techniques to be able to use them alongside DRAM. In this paper, we propose a novel approach for exploiting NVM as a secondary memory partition so that applications can explicitly allocate and manipulate memory regions therein. More specifically, we propose an NVMalloc library with a suite of services that enables applications to access a distributed NVM storage system. We have devised ways within NVMalloc so that the storage system, built from compute node-local NVM devices, can be accessed in a byte-addressable fashion using the memory mapped I/O interface. Our approach has the potential to re-energize out-of-core computations on large-scale machines by having applications allocate certain variables through NVMalloc, thereby increasing the overall memory capacity available. Our evaluation on a 128-core cluster shows that NVMalloc enables applications to compute problem sizes larger than the physical memory in a cost-effective manner. It can bring more performance/efficiency gain with increased computation time between NVM memory accesses or increased data access locality. In addition, our results suggest that while NVMalloc enables transparent access to NVM-resident variables, the explicit control it provides is crucial to optimize application performance.

Computer Science↗

NVMalloc: Exposing an Aggregate SSD Store as a Memory Partition in Extreme-Scale Machines

DRAM is a precious resource in extreme-scale machines and is increasingly becoming scarce, mainly due to the growing number of cores per node. On future multi-petaflop and exaflop machines, the memory pressure is likely to be so severe that we need to rethink our memory usage models. Fortunately, the advent of non-volatile memory (NVM) offers a unique opportunity in this space. Current NVM offerings possess several desirable properties, such as low cost and power efficiency, but suffer from high latency and lifetime issues. We need rich techniques to be able to use them alongside DRAM. In this talk, we present a novel approach for exploiting NVM as a secondary memory partition so that applications can explicitly allocate and manipulate memory regions therein. More specifically, we propose an NVMalloc library with a suite of services that enables applications to access a distributed NVM storage system. We have devised ways within NVMalloc so that the storage system, built from compute node-local NVM devices, can be accessed in a byte-addressable fashion using the memory mapped I/O interface. Our approach has the potential to re-energize out-of-core computations on largescale machines by having applications allocate certain variables through NVMalloc, thereby increasing the overall memory capacity available. Our evaluation on a 128-core cluster shows that NVMalloc enables applications to compute problem sizes larger than the physical memory in a cost-effective manner. It can bring more performance/efficiency gain with increased computation time between NVM memory accesses or increased data access locality. In addition, our results suggest that while NVMalloc enables transparent access to NVM-resident variables, the explicit control it provides is crucial to optimize application performance.

99 GENERAL AND MISCELLANEOUS↗