Search NASA⌕ Search

SEARCH · Search NASA

Results for “Software algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54

Chemist: A Domain-Specific Language by Chemists for Chemists

Managing the complexity of quantum chemistry (QC) software is key to ensuring it remains accessible, maintainable, and reusable. Noticeably missing from the QC ecosystem are modules targeting bottleneck routines. Here we argue that this is likely due to the difficulty in defining interfaces for such modules. To that end, we introduce the open-source, publicly available Chemist library https://github.com/NWChemEx/Chemist. Chemist is a domain-specific language targeting the QC domain. Chemist has been developed focusing on performance and user-friendliness. Using Chemist, QC tasks are defined using familiar domain concepts such as molecules, wave functions, and operators. The domain objects are hierarchical to ensure a systematic encapsulation of information. Key features of Chemist include: extensibility, the ability to alias existing data, and the ability to succinctly define many common QC tasks. The usefulness of Chemist is demonstrated by discussing the interface of NWChemEx’s Fock build module and by showcasing a proof-of-concept self-consistent field algorithm containing uncertainty propagation.

Algorithms↗

A Framework for Lattice QCD Calculations on GPUs

Efficient calculation of the greatest common divisor (GCD) for big integers each whose number of bits is greater than or equal to 1024 has drawn a considerable amount of attention because it can be used to detect a weakness of the RSA security infrastructure. This paper presents a parallel binary GCD algorithm and its implementation for big integers on the Intel Xeon Phi coprocessor. This algorithm is capable of computing GCDs efficiently on many pairs of big integers in parallel by utilizing all cores on a Xeon Phi coprocessor as well as taking advantage of all vector processing units of the coprocessor to speed up critical integer operations within the algorithm. Using 240 threads on a Xeon Phi coprocessor to carry out GCD calculations for a large amount of 2048-bit integers, the implementation achieves the speedup of 30 times over a sequential binary GCD algorithm implementation on a single CPU core, and it delivers twice amount of performance in comparison to the same sequential binary GCD implementation running on 240 threads of the Xeon Phi.

Winter, F. T.↗

Collision of Physics and Software in the Monte Carlo Application Toolkit (MCATK)

The topic is presented in a series of slides organized as follows: MCATK overview, development strategy, available algorithms, problem modeling (sources, geometry, data, tallies), parallelism, miscellaneous tools/features, example MCATK application, recent areas of research, and summary and future work. MCATK is a C++ component-based Monte Carlo neutron-gamma transport software library with continuous energy neutron and photon transport. Designed to build specialized applications and to provide new functionality in existing general-purpose Monte Carlo codes like MCNP, it reads ACE formatted nuclear data generated by NJOY. The motivation behind MCATK was to reduce costs. MCATK physics involves continuous energy neutron & gamma transport with multi-temperature treatment, static eigenvalue (k eff and α) algorithms, time-dependent algorithm, and fission chain algorithms. MCATK geometry includes mesh geometries and solid body geometries. MCATK provides verified, unit-test Monte Carlo components, flexibility in Monte Carlo application development, and numerous tools such as geometry and cross section plotters.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Representing Small Commercial Building Faults in EnergyPlus, Part II: Model Validation

Automated fault detection and diagnosis (AFDD) tools based on machine-learning algorithms hold promise for lowering cost barriers for AFDD in small commercial buildings; however, access to high-quality training data for such algorithms is often difficult to obtain. To fill the gap in this research area, this study covers the development (Part I) and validation (Part II) of fault models that can be used with the building energy modeling software EnergyPlus® and OpenStudio® to generate a cost-effective training data set for developing AFDD algorithms. Part II (this paper) first presents a methodology of validating fault models with OpenStudio and then presents validation results, which are compared against measurements from a reference building. We discuss the results of our experiments with eight different faults in the reference building (a total of 39 different baseline and faulted scenarios), including our methodology for using fault models along with the reference building model to simulate the same faulted scenarios. Then, we present validation of the fault models by comparing results of simulations and experiments either quantitatively or qualitatively.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A non-intrusive parallel-in-time approach for simultaneous optimization with unsteady PDEs

This paper presents a non-intrusive framework for integrating existing unsteady partial differential equation (PDE) solvers into a parallel-in-time simultaneous optimization algorithm. The time-parallelization is provided by the non-intrusive software library XBraid [Parallel multigrid in time, software available at http://llnl.gov/casc/xbraid], which applies an iterative multigrid reduction technique to the time domain of existing time-marching schemes for solving unsteady PDEs. Its general user-interface has been extended in [S. Günther, N.R. Gauger, and J.B. Schroder, A non-intrusive parallel-in-time adjoint solver with the XBraid library, Comput. Vis. Sci. 19 (2018), pp. 85–95. Available at arXiv, math.OC/1705.00663] for computing adjoint sensitivities such that gradients of output quantities with respect to design changes can be computed parallel-in-time alongside with the primal PDE solution. In this paper, the primal and adjoint XBraid iterations are embedded into a simultaneous optimization framework, namely the One-shot method. In this method, design updates towards optimality are employed after each state and adjoint update such that optimality and feasibility of the design and the PDE solution are reached simultaneously. Finally, the time-parallel optimization method is validated on an advection-dominated flow control problem which shows significant speedup over a classical time-serial optimization algorithm.

97 MATHEMATICS AND COMPUTING↗

ADEPT: a domain independent sequence alignment strategy for gpu architectures

Bioinformatic workflows frequently make use of automated genome assembly and protein clustering tools. At the core of most of these tools, a significant portion of execution time is spent in determining optimal local alignment between two sequences. This task is performed with the Smith-Waterman algorithm, which is a dynamic programming based method. With the advent of modern sequencing technologies and increasing size of both genome and protein databases, a need for faster Smith-Waterman implementations has emerged. Multiple SIMD strategies for the Smith-Waterman algorithm are available for CPUs. However, with the move of HPC facilities towards accelerator based architectures, a need for an efficient GPU accelerated strategy has emerged. Existing GPU based strategies have either been optimized for a specific type of characters (Nucleotides or Amino Acids) or for only a handful of application use-cases. In this paper, we present ADEPT, a new sequence alignment strategy for GPU architectures that is domain independent, supporting alignment of sequences from both genomes and proteins. Our proposed strategy uses GPU specific optimizations that do not rely on the nature of sequence. We demonstrate the feasibility of this strategy by implementing the Smith-Waterman algorithm and comparing it to similar CPU strategies as well as the fastest known GPU methods for each domain. ADEPT’s driver enables it to scale across multiple GPUs and allows easy integration into software pipelines which utilize large scale computational systems. We have shown that the ADEPT based Smith-Waterman algorithm demonstrates a peak performance of 360 GCUPS and 497 GCUPs for protein based and DNA based datasets respectively on a single GPU node (8 GPUs) of the Cori Supercomputer. Overall ADEPT shows 10x faster performance in a node-to-node comparison against a corresponding SIMD CPU implementation. ADEPT demonstrates a performance that is either comparable or better than existing GPU strategies. We demonstrated the efficacy of ADEPT in supporting existing bionformatics software pipelines by integrating ADEPT in MetaHipMer a high-performance denovo metagenome assembler and PASTIS a high-performance protein similarity graph construction pipeline. Our results show 10% and 30% boost of performance in MetaHipMer and PASTIS respectively.

59 BASIC BIOLOGICAL SCIENCES↗

WISP: Watching grid Infrastructure Stealthily through Proxies (Final Technical Report)

The complex interdependencies of cyber systems (sensors and communications), physical grids and associated electricity market operations make protecting electric power grids a significant challenge. The energy sector is constantly under new, targeted, advanced and dangerous cyber-attacks that have the potential to result in the loss of human life. These threats are further exacerbated by our need to modernize the grid. One focus of cyber security research in smart grids is the securing of the SCADA system through advanced intrusion detection systems (IDS) and bad data detection algorithms in state estimation. These methods either require full knowledge of the system topology and parameters or fail to understand the physical behaviors under attack. WISP (Watching grid Infrastructure Stealthily through Proxies) is designed to provide additional protection to the power grid using only publicly available data. In particular, WISP exploits the spatio-temporal nature of the real time locational marginal prices (LMPs), in conjunction with other information such as bids, weather, outages and load data to analyze anomalous power pricing behaviors and then correlate those observations to localize regions of interest and identify potential cyber events. WISP is non-intrusive as the tool is deployed as a service in the Cloud or on premise and provides reliable information to system operators for enhanced situational awareness, without impeding energy delivery functions. The WISP technology comprises three modules: the data-driven anomaly detection core, the vulnerability and risk analysis and the root cause analysis. The data-driven anomaly detection core performs the tasks of feature selection, anomaly detection and attack region localization. The vulnerability and risk analysis module provides system level information of the vulnerable variables and times, assisting the operators in selecting monitoring and protection nodes. The root cause analysis module takes the detection results and identifies potential operational conditions that contribute to the detected anomalies. In Phase I, we have demonstrated the feasibility and effectiveness of WISP. We developed a realistic electricity market simulator capable of generating normal and attack market data under various operational conditions. We developed a series of cyber-attack detection and analysis algorithms and evaluated them under multiple data sources. Finally, we integrated all modules into an end-to-end software, providing functions for data management, data analytics and visualization. Specifically, we have achieved: (i) real-time data acceptance from external utility interfaces with >99% acceptance rate; (ii) high performance anomaly detection algorithms with >98% detection accuracy and <0.1% false alarm rate; and (iii) ultra-low computing delay <50 milliseconds. Additionally, our team developed algorithms to identify the vulnerable variables in electricity market operations and root cause analysis functions to identify major contributors to the price spikes. These ancillary modules are necessary when deploying WISP in real world industry environment. In Phase II, we have demonstrated the effectiveness of WISP software on realistic largescale power systems. We performed red team testing for the Phase I WISP software and identified software vulnerabilities and implemented corresponding mitigation solutions. We adapted the electricity market simulator for the Texas synthetic 2000-bus system and generated datasets for the false data injection attacks. We created database and visualization interfaces for the Texas system and the ISO New England system. We performed software optimization in terms of operation efficiency, computing speed and detection accuracy. Finally, we tested the software on the Texas system and the ISO New England system and evaluated the detection performance. Overall, we achieved above 89% detection rate, below 3% false alarm rate and below 37 seconds of end-to-end detection delay.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Invited: Software defined accelerators from learning tools environment

Next generation systems, such as edge devices, will need to provide efficient processing of machine learning (ML) algorithms along several metrics, including energy, performance, area, and latency. However, the quickly evolving field of ML makes it extremely difficult to generate accelerators able to support a wide variety of algorithms. At the same time, designing accelerators in hardware description languages (HDLs) by hand is hard and time consuming, and does not allow quick exploration of the design space. In this paper we present the Software Defined Accelerators From Learning Tools Environment (SODALITE), an automated open source high-level ML framework-to-verilog compiler targeting ML Application-Specific Integrated Circuits (ASICs) chiplets. The SODALITE approach will implement optimal designs by seamlessly combining custom components generated through high-level synthesis (HLS) with templated and fully tunable Intellectural Properties (IPs) and macros, integrated in an extendable resource library. Through a closed loop design space exploration engine, developers will be able to quickly explore their hardware designs along different dimensions.

High-Level Synthesis, Accelerators, Hardware-Softw↗

Refinements and Tests of an Advanced Controller to Mitigate Fatigue Loads in the Controls Advanced Research Turbine: Preprint

Wind turbines are complex, nonlinear, dynamic systems forced by aerodynamic, gravitational, centrifugal, and gyroscopic loads. The aerodynamics of wind turbines are nonlinear, unsteady, and complex. Turbine rotors are subjected to a complicated 3-D turbulent wind inflow field, with imbedded coherent vortices that drive fatigue loads and reduce lifetime. Design of control algorithms for windturbines must account for multiple control objectives. Future large multi-megawatt turbines must be designed with lighter weight structures, using active controls to mitigate fatigue loads, while maximizing energy capture. Active damping should be added to these dynamic structures to maintain stability for operation in a complex environment. At the National Renewable Energy Laboratory (NREL), wehave designed, implemented, and tested advanced controls to maximize energy extraction and reduce structural dynamic loads. These control designs are based on linear models of the turbine that are generated by specialized modeling software. In this paper, we present field test results of an advanced control algorithm to mitigate blade, tower, and drivetrain loads in Region 3.

17 WIND ENERGY↗

The Suite for Embedded Applications and Kernels

Many applications of high performance embedded computing are limited by performance or power bottlenecks. We have designed SEAK, a new benchmark suite, (a) to capture these bottlenecks in a way that encourages creative solutions to these bottlenecks? and (b) to facilitate rigorous, objective, end-user evaluation for their solutions. To avoid biasing solutions toward existing algorithms, SEAK benchmarks use a mission-centric (abstracted from a particular algorithm) and goal-oriented (functional) specification. To encourage solutions that are any combination of software or hardware, we use an end-user blackbox evaluation that can capture tradeoffs between performance, power, accuracy, size, and weight. The tradeoffs are especially informative for procurement decisions. We call our benchmarks future proof because each mission-centric interface and evaluation remains useful despite shifting algorithmic preferences. It is challenging to create both concise and precise goal-oriented specifications for mission-centric problems. This paper describes the SEAK benchmark suite and presents an evaluation of sample solutions that highlights power and performance tradeoffs.

Source record↗

Alfalfa [SWR 18-17]

Alfalfa is a web based software service that implements a virtual building control system using the EnergyPlus simulation as a backend. Conventional building energy simulation is non interactive, meaning that a simulation is started and run to completion without any user input during the simulation. Alfalfa allows a client to start a building simulation, then interactively read sensor locations and dynamically command actuators in realtime. The Alfalfa web service conforms to the Project Haystack standard, which is a specification adopted by many building control system vendors. The service enables Haystack compliant control system vendors to test their control modules and algorithms without the need for a physical building or modifying their hardware and software, enabling faster and cheaper development cycles. Alfalfa has use cases beyond control system testing. One example is building fault detection, where physical building performance can be compared to simulated values in realtime, indicating where there may be a problem in the building. Another use case is informing predictive control systems, where near future events are simulated to premptively guide control decisions in the real building. This software is web based and designed to be self service. Users of Alfalfa can upload an OpenStudio building model and begin communicating with the simulated building without any advanced setup or configuration, as if they were communicating a real building control system. Behind the scenes, Alfalfa makes extensive use of Amazon Web Services, in order to scale from a very small number of users and buildings, to extremely large numbers in a cost effective way.

Benne, Kyle↗

Hiperclust

This software leverages transfer learning to analyze atom probe tomography (APT) data. It is trained on synthetic data and then applies this knowledge to predict the optimal number of clusters for a given APT dataset. Initially, the software used preliminary clustering to estimate the general structure of the data. Based on this, it provides suggestions for key parameters like minimum cluster size and minimum number of points. These parameters are critical for algorithms like HDBSCAN, ensuring accurate cluster formation without the need for trial-and-error testing. The software runs on High-Performance computing (HPC) systems, enabling fast, scalable analysis of large APT datasets, ultimately saving time and improving the reliability of clustering outcomes.

Tang, Yalei [Idaho National Laboratory (INL), Idah↗

SU-E-J-115: Correlation of Displacement Vector Fields Calculated by Deformable Image Registration Algorithms with Motion Parameters of CT Images with Well-Defined Targets and Controlled-Motion

Purpose: To investigate correlation of displacement vector fields (DVF) calculated by deformable image registration algorithms with motion parameters in helical axial and cone-beam CT images with motion artifacts. Methods: A mobile thorax phantom with well-known targets with different sizes that were made from water-equivalent material and inserted in foam to simulate lung lesions. The thorax phantom was imaged with helical, axial and cone-beam CT. The phantom was moved with a cyclic motion with different motion amplitudes and frequencies along the superior-inferior direction. Different deformable image registration algorithms including demons, fast demons, Horn-Shunck and iterative-optical-flow from the DIRART software were used to deform CT images for the phantom with different motion patterns. The CT images of the mobile phantom were deformed to CT images of the stationary phantom. Results: The values of displacement vectors calculated by deformable image registration algorithm correlated strongly with motion amplitude where large displacement vectors were calculated for CT images with large motion amplitudes. For example, the maximal displacement vectors were nearly equal to the motion amplitudes (5mm, 10mm or 20mm) at interfaces between the mobile targets lung tissue, while the minimal displacement vectors were nearly equal to negative the motion amplitudes. The maximal and minimal displacement vectors matched with edges of the blurred targets along the Z-axis (motion-direction), while DVF’s were small in the other directions. This indicates that the blurred edges by phantom motion were shifted largely to match with the actual target edge. These shifts were nearly equal to the motion amplitude. Conclusions: The DVF from deformable-image registration algorithms correlated well with motion amplitude of well-defined mobile targets. This can be used to extract motion parameters such as amplitude. However, as motion amplitudes increased, image artifacts increased significantly and that limited image quality and poor correlation between the motion amplitude and DVF was obtained.

60 APPLIED LIFE SCIENCES↗

Batched matrix computations on hardware accelerators based on GPUs

Scientific applications require solvers that work on many small size problems that are independent from each other. At the same time, the high-end hardware evolves rapidly and becomes ever more throughput-oriented and thus there is an increasing need for an effective approach to develop energy-efficient, high-performance codes for these small matrix problems that we call batched factorizations. The many applications that need this functionality could especially benefit from the use of GPUs, which currently are four to five times more energy efficient than multicore CPUs on important scientific workloads. This study, consequently, describes the development of the most common, one-sided factorizations, Cholesky, LU, and QR, for a set of small dense matrices. The algorithms we present together with their implementations are, by design, inherently parallel. In particular, our approach is based on representing the process as a sequence of batched BLAS routines that are executed entirely on a GPU. Importantly, this is unlike the LAPACK and the hybrid MAGMA factorization algorithms that work under drastically different assumptions of hardware design and efficiency of execution of the various computational kernels involved in the implementation. Thus, our approach is more efficient than what works for a combination of multicore CPUs and GPUs for the problems sizes of interest of the application use cases. The paradigm where upon a single chip (a GPU or a CPU) factorizes a single problem at a time is not at all efficient in our applications’ context. We illustrate all of these claims through a detailed performance analysis. With the help of profiling and tracing tools, we guide our development of batched factorizations to achieve up to two-fold speedup and three-fold better energy efficiency as compared against our highly optimized batched CPU implementations based on MKL library. Finally, the tested system featured two sockets of Intel Sandy Bridge CPUs and we compared with a batched LU factorizations featured in the CUBLAS library for GPUs, we achieve as high as 2.5× speedup on the NVIDIA K40 GPU.

97 MATHEMATICS AND COMPUTING↗

Algorithm and Architecture Independent Benchmarking with SEAK

Many applications of high performance embedded computing are limited by performance or power bottlenecks. We have designed the Suite for Embedded Applications & Kernels (SEAK), a new benchmark suite, (a) to capture these bottlenecks in a way that encourages creative solutions; and (b) to facilitate rigorous, objective, end-user evaluation for their solutions. To avoid biasing solutions toward existing algorithms, SEAK benchmarks use a mission-centric (abstracted from a particular algorithm) and goal-oriented (functional) specification. To encourage solutions that are any combination of software or hardware, we use an end-user black-box evaluation that can capture tradeoffs between performance, power, accuracy, size, and weight. The tradeoffs are especially informative for procurement decisions. We call our benchmarks future proof because each mission-centric interface and evaluation remains useful despite shifting algorithmic preferences. It is challenging to create both concise and precise goal-oriented specifications for mission-centric problems. This paper describes the SEAK benchmark suite and presents an evaluation of sample solutions that highlights power and performance tradeoffs.

algorithms and kernels↗

SU-F-J-98: Improvement and Evaluation Of Deformation Image Registration On Parotid Glands During Radiation Therapy for Nasopharyngeal Cancer

Purpose: To quantitatively evaluate the strategic innovation and accuracy variation of deformation registration algorithm for parotid glands using the similarity Dice coefficient during the course of radiation therapy (RT) for nasopharyngeal cancer (NPC). Methods: Daily MVCT data for 10 patients with pathologically proven nasopharyngeal cancers were analyzed. The data were acquired using tomotherapy (TomoTherapy, Accuray) at the PLA General Hospital. The prescription dose to the primary target was 70Gy in 33 fractions. Two kinds of contours for parotid glands on daily MVCTs were obtained by populating these contours from planning CTs to the daily CTs via rigid-body registration with or without the rotation shifts using the in-house tools and the Adaptive plan software (Adaptive Plan, TomoTherapy), and were edited manually if necessary. The diffeomorphic Demons algorithm developed in the in-house tool was used to propagate the parotid structures from the daily CTs to planning CTs. The differences of the mapped parotid contours in two methods were evaluated using Dice similarity index (DSI). Two-tailed t-test analysis was carried out to compare the DSI changes during the course of RT. Results: For 10 patient plans, the accuracy of deformation image registration (DIR) with the rotation shift was obviously better than those without the rotation shift. The Dice scores of the ipsi- and contra-lateral parotids for with and without the rotation shifts were found to be correlated with each other [0.904±0.031 vs 0.919±0.030 (p<0.001); 0.900±0.031 vs 0.910±0.032 (p<0.001)]. The Dice scores for the parotids have shown the reduction with the changes of parotid volumes during RT. The DSI values between the first and last fraction were 0.932±0.020 vs 0.899±0.030 in 10 patient plans. Conclusion: DIR was successfully improved using the strategic innovation for ART. And the decrease of DIR accuracy has also been found during the delivery of fractionated radiotherapy. This work was supported in part by the grant from Chinese Natural Science Foundation (Grant No. 11105225).

60 APPLIED LIFE SCIENCES↗

Search for New Phenomena Using W/Z + (b)-Jets Measurements Performed with the ATLAS Detector

The Project proposed to use data of the ATLAS experiment, obtained during the 2011 and 2012 data-taking campaigns, to pursue studies of the strong interaction (QCD) and to examine promising signatures for new physics. The Project also contains a service component dedicated to a detector development initiative. The objective of the strong interaction studies is to determine how various predictions from the main theory (QCD) compare to the data. Results of a set of measurements developed by the Tufts team indicate that the dominant factor of discrepancy between data and QCD predictions come from the mis-modeling of the low energy gluon radiation as described by algorithms called parton showers. The discrepancies introduced by parton showers on LHC predictions could even be larger than the effect due to completely new phenomena (dark matter, supersymmetry, etc.) and could thus block further discoveries at the LHC. Some of the results obtained in the course of this Project also specify how QCD predictions must be improved in order to open the possibility for the discovery of something completely new at the LHC during Run-II. This has been integrated in the Run-II ATLAS physics program. Another objective of Tufts studies of the strong interaction was to determine how the hypothesis about an intrinsic heavy-quark component of the proton (strange, charm or bottom quarks) could be tested at the LHC. This hypothesis has been proposed by theorists 30 years ago and is still controversial. The Tufts team demonstrated that intrinsic charms can be observed, or severely constrained, at the LHC, and determine how the measurement should be performed in order to maximize its sensitivity to such an intrinsic heavy-quark component of the proton. Tufts also embarked on performing the measurement that is in progress, but final results are not yet available. They should shade a light of understanding on the fundamental structure of the proton. Determining the nature of dark matter particles, composing about 25% of all the matter in the universe, is one of the most exciting research goals at the LHC. Within this Project, the Tufts team proposed a way to improve over the standard approach used to look for dark matter at the LHC in events involving jets and a large amount of unbalanced energy in the detector (jets+ETmiss). The Tufts team has developed a measurement to test these improvements on data available (ATLAS 2012 dataset), in order to be ready to apply them on the new Run-II data that will be available at the end of 2015. Preliminary results on the proposed measurement indicate that a very high precision can be obtained on results free of detector effects. That will allow for better constrains of dark matter theories and will spare the needs for huge computing resources in order to compare dark matter theories to data. Finally, the Tufts team played a leading role in the development and the organization of the 6Et trigger, the detector component needed to collect the data used in dark matter searches and in many other analyses. The team compared the performance of the various algorithms capable of reconstructing the value of the ETmiss on each LHC collision event, and developed a strategy to commission these algorithms online. Tufts also contributed in the development of the ETmiss trigger monitoring software. Finally, the PI of this Project acted as the co-coordinator of the group of researchers at CERN taking care of the development and the operation of this detector component. The ETmiss trigger is now taking data, opening the possibility for the discovery of otherwise undetectable particles at the LHC.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

PROTEUS: Machine Learning Driven Resilience for Extreme-scale Systems

The objective of this project is to design, develop, and evaluate scalable software to enhance resilience, data checkpointing, program restart, and analysis. The proposed tasks are to 1) develop scalable machine learning techniques to learn temporal change patterns in a scalable and in-situ manner, and to minimize data movement and maximize learning locally closest to data; 2) design a concise data representation and indexing mechanism to capture the distribution of changes in data that can guarantee point-wise user-defined tolerable errors while reducing the data storage requirements by an order of magnitude or more; 3) develop data reduction techniques as library modules; 4) exploit local SSD for minimizing data movement in storage hierarchy; 5) develop anomaly detection algorithms that can predict corruptions based on learning of emerging patterns; 6) develop software libraries to be incorporated within widely used data formats and APIs; and 7) evaluate the proposed software using DOE scientific applications. The outcomes of the proposed work are to satisfy many synergistic data reduction and resilience requirements for large-scale data intensive applications executed on extreme-scale computing systems. The developed mechanism for error-bound data approximation is directly applicable to existing scientific applications. Through machine learning from historical events and change distribution, this work will enable anomaly detection for DOE computer facility.

97 MATHEMATICS AND COMPUTING↗