Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel intelligence”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS1996

On-board Neural Processor Design for an Intelligent Multi-sensor Microspacecraft

A compact VLSI neural processor based on the Optimization Cellular Neural Network (OCNN)has been under development to provide a wide range of support for an intelligent remote sensing microspacecraft which requires both high bandwidth communication and high-performance computing for on-board data analysis, thematic data reduction, synergy of multiple types of sensors, and other smart-sensor functions. The OCNN architecture is a programmable multi-dimensional array of neurons which are locally connected with their local neurons. The OCNN operation theory, architecture, design and implementation, prototype chip, and system applications have been investigated in detail and presented in this paper.

array↗
NASA NTRS1990

Architectural study of the design and operation of advanced force feedback manual controllers

A teleoperator system consists of a manual controller, control hardware/software, and a remote manipulator. It was employed in either hazardous or unstructured, and/or remote environments. In teleoperation, the main-in-the-loop is the central concept that brings human intelligence to the teleoperator system. When teleoperation involves contact with an uncertain environment, providing the feeling of telepresence to the human operator is one of desired characteristics of the teleoperator system. Unfortunately, most available manual controllers in bilateral or force-reflecting teleoperator systems can be characterized by their bulky size, high costs, or lack of smoothness and transparency, and elementary architectures. To investigate other alternatives, a force-reflecting, 3 degree of freedom (dof) spherical manual controller is designed, analyzed, and implemented as a test bed demonstration in this research effort. To achieve an improved level of design to meet criteria such as compactness, portability, and a somewhat enhanced force-reflecting capability, the demonstration manual controller employs high gear-ratio reducers. To reduce the effects of the inertia and friction on the system, various force control strategies are applied and their performance investigated. The spherical manual controller uses a parallel geometry to minimize inertial and gravitational effects on its primary task of transparent information transfer. As an alternative to the spherical 3-dof manual controller, a new conceptual (or parallel) spherical 3-dof module is introduced with a full kinematic analysis. Also, the resulting kinematic properties are compared to those of other typical spherical 3-dof systems. The conceptual design of a parallel 6-dof manual controller and its kinematic analysis is presented. This 6-dof manual controller is similar to the Stewart Platform with the actuators located on the base to minimize the dynamic effects. Finally, a combination of the new 3-dof and 6-dof concepts is presented as a feasible test-bed for enhanced performance in a 9-dof system.

Tesar, Delbert↗
DOE OSTI2025

Integration of multiple coinflip devices for high-quality random sampling

Artificial intelligence, scientific computing, and probabilistic computing use random sampling to approximate solutions to various problems, with larger models requiring a substantial quantity of random numbers. To generate the required vast quantity of random numbers at high rates, we explore so-called “coinflip” devices, which are stochastic microelectronic devices ideally capable of independently generating random bits with a tunable weight at a high rate. However, coinflip devices are inherently analog and demonstrate nonidealities, like temperature dependence and drift, that can introduce determinism into the outputs. We present important considerations for building systems of multiple coinflip devices to produce high-quality bitstreams with low error and little dependency on previous bits. Using tunnel diodes as coinflip devices, we implement a control loop to adapt to temperature dependence and generate fair bitstreams with each device. While this can lead to dependencies between bits in a single bitstream, we demonstrate that combining results generated in parallel with individual tunnel diodes can produce fair and unpredictable bitstreams. The suitability of these bitstreams for use in probabilistic computing is then demonstrated through a Monte Carlo approximation of π.

Taylor, Brady Garland [Sandia National Laboratorie↗
DOE OSTI2024

AGS-GNN: Attribute-guided Sampling for Graph Neural Networks

We propose AGS-GNN, a novel attribute-guided sampling algorithm for Graph Neural Networks (GNNs) that exploits node features and connectivity structure of a graph while simultaneously adapting for both homophily and heterophily in graphs. (In homophilic graphs vertices of the same class are more likely to be connected, and vertices of different classes tend to be linked in heterophilic graphs.) While GNNs have been successfully applied to homophilic graphs, their application to heterophilic graphs remains challenging. The best-performing GNNs for heterophilic graphs do not fit the sampling paradigm, suffer high computational costs, and are not inductive. We employ samplers based on feature-similarity and feature-diversity to select subsets of neighbors for a node, and adaptively capture information from homophilic and heterophilic neighborhoods using dual channels. Currently, AGS-GNN is the only algorithm that we know of that explicitly controls homophily in the sampled subgraph through similar and diverse neighborhood samples. For diverse neighborhood sampling, we employ submodularity, which was not used in this context prior to our work. The sampling distribution is pre-computed and highly parallel, achieving the desired scalability. Using an extensive dataset consisting of 35 small (<=100K nodes) and large (>100K nodes) homophilic and heterophilic graphs, we demonstrate the superiority of AGS-GNN compare to the current approaches in the literature. AGS-GNN achieves comparable test accuracy to the best-performing heterophilic GNNs, even outperforming methods using the entire graph for node classification. AGS-GNN also converges faster compared to methods that sample neighborhoods randomly, and can be incorporated into existing GNN models that employ node or graph sampling.

artificial intelligence↗
DOE OSTI2021

Foundational Open Source Solar System Modeling Through Improvement and Validation of the System Advisor Model and PVWatts (FY19-FY21 Final Technical Report)

Accelerating intelligent deployment of solar energy technologies demands accurate system modeling every step of the way. From project development to policy research, grid integration studies to development of novel technologies, industry and researchers alike need validated, transparent, easy-to-use, extensible, cutting-edge, and accurate models of both the performance and financing of solar systems. The System Advisor Model (SAM) and PVWatts tools provide a platform to fill that need. The overarching goal of this set of software tools is to enable accurate PV system modeling across the industry, and our usage metrics indicate that we continue to succeed in that endeavor, with a user starting SAM every 2 minutes globally, and over 17 million PVWatts hits per month. This project leveraged DOEs past investment in the SAM and PVWatts platforms to continue to provide valuable and extensible PV, battery, and financial modeling resources to the larger solar community. We pursued multiple avenues in parallel: software maintenance and technical support that are foundational to the continued usability of the SAM and PVWatts platforms; platform and PV model improvements and stakeholder engagement activities that are core to the continued relevance of the platforms; and open source activities to foster the continued creation of a vibrant open-source community around the SAM and PVWatts tools, which opens up exciting new opportunities for industry interaction.

14 SOLAR ENERGY↗
NASA NTRS1989

Identification and interpretation of patterns in rocket engine data: Artificial intelligence and neural network approaches

This paper describes an expert system which is designed to perform automatic data analysis, identify anomalous events, and determine the characteristic features of these events. We have employed both artificial intelligence and neural net approaches in the design of this expert system. The artificial intelligence approach is useful because it provides (1) the use of human experts' knowledge of sensor behavior and faulty engine conditions in interpreting data; (2) the use of engine design knowledge and physical sensor locations in establishing relationships among the events of multiple sensors; (3) the use of stored analysis of past data of faulty engine conditions; and (4) the use of knowledge-based reasoning in distinguishing sensor failure from actual faults. The neural network approach appears promising because neural nets (1) can be trained on extremely noisy data and produce classifications which are more robust under noisy conditions than other classification techniques; (2) avoid the necessity of noise removal by digital filtering and therefore avoid the need to make assumptions about frequency bands or other signal characteristics of anomalous behavior; (3) can, in effect, generate their own feature detectors based on the characteristics of the sensor data used in training; and (4) are inherently parallel and therefore are potentially implementable in special-purpose parallel hardware.

Ali, Moonis↗
DOE OSTI2025

Global tuning of hadronic interaction models with accelerator-based and astroparticle data

In high-energy and astroparticle physics, event generators play an essential role, even in the simplest data analyses. As analysis techniques become more sophisticated, e.g. based on deep neural networks, their correct description of the observed event characteristics becomes even more important. Physical processes occurring in hadronic collisions are simulated within a Monte Carlo framework. A major challenge is the modeling of hadron dynamics at low momentum transfer, which includes the initial and final phases of every hadronic collision. QCD-inspired phenomenological models used for these phases cannot guarantee completeness or correctness over the full phase space. These models usually include parameters which must be tuned to suitable experimental data. Until now, event generators have been developed and tuned mainly on the basis of data from high-energy physics experiments at accelerators. The wealth of data available from the latest generation of astroparticle experiments has not yet been fully exploited, and in many cases is not satisfactorily described. Both kinds of data sets are complementary as astroparticle experiments provide sensitivity especially to hadrons produced nearly parallel to the collision axis and cover center-of-mass energies up to several hundred TeV, well beyond those reached at colliders so far. In this report, we provide an overview of state-of-the-art event generators and their tuning, including the most relevant inputs from high-energy accelerator and astroparticle experiments. We present a road map that shows, for the first time, how the unified tuning of event generators with accelerator-based and astroparticle data can be performed.

Albrecht, J. [Ruhr U., Bochum, RAPP Ctr.; Ruhr U.,↗
DOE OSTI2026

Explainable machine learning for incipient anomaly detection in compact molten salt heat exchanger with overlapping feature distributions

High-temperature molten salt-cooled reactors (MSCRs) are a promising next-generation nuclear technology option, offering efficient power conversion and inherent safety features. However, the reliability of these systems depends on the robust operation of heat exchangers (HXs), which are susceptible to failure due to temperature gradients and channel plugging caused by fluid freezing. Conventional monitoring methods, relying on inlet and outlet measurements, lack the spatial resolution needed to detect early-stage faults. We propose a novel design of a compact salt-to-salt matrix-type HX design consisting of interleaved arrays of parallel tubes, with integrated synthetic fiber optic distributed temperature sensing (DTS) to enable localized detection of incipient faults. To evaluate performance of this design, we generate high-fidelity synthetic data using heat transfer computational modeling to simulate channel plugging, and introduce sensor noise for realistic modeling of measurements. The dataset comprises of 97% normal operation and 3% anomaly cases, with each anomaly class representing 1% of the data. These early anomalies result in overlapping temperature profiles between normal and faulty channels, producing a non-separable dataset that challenges traditional classification techniques. We benchmark eight supervised machine learning (ML) models and demonstrate that XGBoost achieves the highest performance. To improve transparency, we develop an explainability framework combining Shapley values and partially ordered sets (POSETs) to quantify and structurally analyze feature importance. This approach identifies both dominant predictors and ambiguous feature relationships, enhancing trust and interpretability. Our results highlight the potential of combining DTS and explainable ML with intelligent feature selection to improve predictive maintenance and ensure operational resilience in advanced nuclear systems.

Prantikos, Konstantinos [Argonne National Laborato↗
NASA NTRS1990

Artificial Intelligence (AI) Based Tactical Guidance for Fighter Aircraft

A research program investigating the use of Artificial Intelligence (AI) techniques to aid in the development of a Tactical Decision Generator (TDG) for Within Visual Range (WVR) air combat engagements is discussed. The application of AI programming and problem solving methods in the development and implementation of the Computerized Logic For Air-to-Air Warfare Simulations (CLAWS), a second generation TDG, is presented. The Knowledge-Based Systems used by CLAWS to aid in the tactical decision-making process are outlined in detail, and the results of tests to evaluate the performance of CLAWS versus a baseline TDG developed in FORTRAN to run in real-time in the Langley Differential Maneuvering Simulator (DMS), are presented. To date, these test results have shown significant performance gains with respect to the TDG baseline in one-versus-one air combat engagements, and the AI-based TDG software has proven to be much easier to modify and maintain than the baseline FORTRAN TDG programs. Alternate computing environments and programming approaches, including the use of parallel algorithms and heterogeneous computer networks are discussed, and the design and performance of a prototype concurrent TDG system are presented.

McManus, John W.↗
DOE OSTI2025

Design, Optimization, and Control of Floating Offshore Wind Farms for Optimal Energy Production (Final report)

The uncertainty and irregularity of ocean waves and the ocean environment is a major factor in the development of commercial scale floating wind turbines as the operation of floating structures in such an environment can lead to irregular and unpredictable loading, fatigue, and ultimately a reduction in the operational life of the turbine system which affects energy production over the lifetime of the turbine. Control solutions that can limit float motions and mitigate stressful events on the structure become essential for extending lifetime and limiting the operational uncertainty of a floating wind turbine. Digital twins are computational replicas of physical systems that operate in parallel with the operation of the physical system. Given advanced knowledge of a systems input, digital twins have the ability to predict the behavior of a system in advance, which can be valuable in the control of that system. In this project, we developed and assessed potential digital twin models developed in house and openly available (OpenFast) for use in the real time control of the six degree of freedom response motions of a floating wind turbine in ocean waves. Coupling these models with near-field real time irregular sea surface (wve) measurement/sensings and prediction models, we used the digital twin to predict how the floating turbine will respond to the incoming waves. Applying this information to a motion control system of the float, one can limit and control float motions to prevent undesirable loading events/large angular motions, thus increasing system life and ultimately contributing to optimizing energy production. Due to the computational intensity of operating a digital twin in real time, we investigated the use of artificial intelligence techniques to speed-up the processes of the digital twin, as well as the wave reconstruction/prediction models. Model tank testing at the University of Rhode Island and University of Maine both validated and demonstrated the developed techniques on simple float geometries and a scale model of the NREL 15 MW reference turbine.

17 WIND ENERGY↗
DOE OSTI2020

Machine Committee Framework for Power Grid Disturbances Analysis Using Synchrophasors Data

Events detection is a key challenge in power grid frequency disturbances analysis. Accurate detection of events is crucial for situational awareness of the power system. In this paper, we study the problem of events detection in power grid frequency disturbance analysis using synchrophasors data streams. Current events detection approaches for power grid rely on individual detection algorithm. This study integrates some of the existing detection algorithms using the concept of machine committee to develop improved detection approaches for grid disturbance analysis. Specifically, we propose two algorithms—an Event Detection Machine Committee (EDMC) algorithm and a Change-Point Detection Machine Committee (CPDMC) algorithm. Both algorithms use parallel architecture to fuse detection knowledge of its individual methods to arrive at an overall output. The EDMC algorithm combines five individual event detection methods, while the CPDMC algorithm combines two change-point detection methods. Each method performs the detection task separately. The overall output of each algorithm is then computed using a voting strategy. The proposed algorithms are evaluated using three case studies of actual power grid disturbances. Compared with the individual results of the various detection methods, we found that the EDMC algorithm is a better fit for analyzing synchrophasors data; it improves the detection accuracy; and it is suitable for practical scenarios.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Artificial Intelligence for Multiphysics Nuclear Design Optimization with Additive Manufacturing

The geometric flexibility of additively manufactured metals and ceramics generates a very large and open design space that requires advanced modeling and simulation tools for physics simulations and the rigorous definition of design problems. This effort deploys artificial intelligence (AI) and machine learning (ML) algorithms to understand the design space, evaluate potential designs, and more efficiently generate optimized results. The Transformational Challenge Reactor (TCR) program is leveraging advances in several scientific areas—including materials, manufacturing, sensors and control systems, data analytics, and high-fidelity modeling and simulation—to accelerate the design, manufacturing, qualification, and deployment of advanced nuclear energy systems. Through a manufacturing-informed design approach, the TCR program seeks to integrate digital data for rapid nuclear innovation; accelerate the adoption of advances in manufacturing, materials, and computational sciences for nuclear applications; and dramatically reduce deployment costs and timelines for new nuclear reactor technologies. This report documents efforts under the TCR program to leverage advanced modeling and simulation techniques driven by AI/ML algorithms on high-performance computing (HPC) systems to yield more optimized TCR core designs. A multiphysics ML surrogate model was developed to run on the HPC architectures. The surrogate model is trained on high-fidelity simulation data of coupled neutronics and thermofluidics and is used to quickly evaluate thousands of candidate core designs in parallel, which drives the evolution of the cooling channel shapes to minimize temperature peaking and material stress. Outcomes from these activities provide design information and feedback into the core design efforts.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Expert and operator perspectives on barriers to energy efficiency in data centers

Abstract It was last estimated in 2016 that data centers (DCs) comprise approximately 2% of total US electricity consumption. However, this estimate is currently being updated to account for the massive increase in computing needs due to streaming, cryptocurrency, and artificial intelligence (AI). To prevent energy consumption that tracks with increasing computing needs, it is imperative we identify energy efficiency strategies and investments beyond the low-hanging fruit solutions. In a two-phased research approach, we ask: What non-technical barriers still impede energy efficiency (EE) practices and investments in the data center sector, and what can be done to overcome these barriers? In particular, we are focused on social and organizational barriers to EE. In Phase I, we performed a literature review and found that technical solutions are abundant in the literature, but fail to address the top-down cultural shifts that need to take place in order to adapt new energy efficiency strategies. In Phase II, reported here, we interviewed 16 data center operators/experts to ground-truth our literature findings. Our interview protocols focus on three aspects of DC decision-making: procurement practices, metrics and monitoring, and perceived barriers to energy efficiency. We find that vendors are the key drivers of procurement decisions, advanced efficiency metrics are facility-specific, and there is convergence in the design of advanced facilities due to the heat density of parallelized infrastructure. Our ultimate goals for our research are to design DC decarbonization policies that target organizational structure, empower individual staff, and foster a supportive external market.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2016

Detector Control and Data Acquisition for the Wide-Field Infrared Survey Telescope (WFIRST) with a Custom ASIC

The Wide-Field Infrared Survey Telescope (WFIRST) will have the largest near-IR focal plane ever flown by NASA, a total of 18 4K x 4K devices. The project has adopted a system-level approach to detector control and data acquisition where 1) control and processing intelligence is pushed into components closer to the detector to maximize signal integrity, 2) functions are performed at the highest allowable temperatures, and 3) the electronics are designed to ensure that the intrinsic detector noise is the limiting factor for system performance. For WFIRST, the detector arrays operate at 90 to 100 K, the detector control and data acquisition functions are performed by a custom ASIC at 150 to 180 K, and the main data processing electronics are at the ambient temperature of the spacecraft, notionally approx.300 K. The new ASIC is the main interface between the cryogenic detectors and the warm instrument electronics. Its single-chip design provides basic clocking for most types of hybrid detectors with CMOS ROICs. It includes a flexible but simple-to-program sequencer, with the option of microprocessor control for more elaborate readout schemes that may be data-dependent. All analog biases, digital clocks, and analog-to-digital conversion functions are incorporated and are connected to the nearby detectors with a short cable that can provide thermal isolation. The interface to the warm electronics is simple and robust through multiple LVDS channels. It also includes features that support parallel operation of multiple ASICs to control detectors that may have more capability or requirements than can be supported by a single chip.

data acquisition↗
DOE OSTI2025

Communication Lower Bounds and Optimal Algorithms for Symmetric Matrix Computations

In this article, we focus on the communication costs of three symmetric matrix computations: (i) multiplying a matrix with its transpose, known as a symmetric rank-k update (SYRK) (ii) adding the result of the multiplication of a matrix with the transpose of another matrix and the transpose of that result, known as a symmetric rank-2k update (SYR2K) (iii) performing matrix multiplication with a symmetric input matrix (SYMM). All three computations appear in the Level 3 Basic Linear Algebra Subroutines (BLAS) and have wide use in applications involving symmetric matrices. We establish communication lower bounds for these kernels using sequential and distributed-memory parallel computational models, and we show that our bounds are tight by presenting communication-optimal algorithms for each setting. Our lower bound proofs rely on applying a geometric inequality for symmetric computations and analytically solving constrained nonlinear optimization problems. As a result, the symmetric matrix and its corresponding computations are accessed and performed according to a triangular block partitioning scheme in the optimal algorithms.

Al Daas, Hussam [Rutherford Appleton Laboratory, D↗
DOE OSTI2022

Decoding the protein–ligand interactions using parallel graph neural networks

Abstract Protein–ligand interactions (PLIs) are essential for biochemical functionality and their identification is crucial for estimating biophysical properties for rational therapeutic design. Currently, experimental characterization of these properties is the most accurate method, however, this is very time-consuming and labor-intensive. A number of computational methods have been developed in this context but most of the existing PLI prediction heavily depends on 2D protein sequence data. Here, we present a novel parallel graph neural network (GNN) to integrate knowledge representation and reasoning for PLI prediction to perform deep learning guided by expert knowledge and informed by 3D structural data. We develop two distinct GNN architectures: $$\hbox {GNN}_{\mathrm{F}}$$ GNN F is the base implementation that employs distinct featurization to enhance domain-awareness, while $$\hbox {GNN}_{\mathrm{P}}$$ GNN P is a novel implementation that can predict with no prior knowledge of the intermolecular interactions. The comprehensive evaluation demonstrated that GNN can successfully capture the binary interactions between ligand and protein’s 3D structure with 0.979 test accuracy for $$\hbox {GNN}_{\mathrm{F}}$$ GNN F and 0.958 for $$\hbox {GNN}_{\mathrm{P}}$$ GNN P for predicting activity of a protein–ligand complex. These models are further adapted for regression tasks to predict experimental binding affinities and $$\hbox {pIC}_{\mathrm{50}}$$ pIC 50 crucial for compound’s potency and efficacy. We achieve a Pearson correlation coefficient of 0.66 and 0.65 on experimental affinity and 0.50 and 0.51 on $$\hbox {pIC}_{\mathrm{50}}$$ pIC 50 with $$\hbox {GNN}_{\mathrm{F}}$$ GNN F and $$\hbox {GNN}_{\mathrm{P}}$$ GNN P , respectively, outperforming similar 2D sequence based models. Our method can serve as an interpretable and explainable artificial intelligence (AI) tool for predicted activity, potency, and biophysical properties of lead candidates. To this end, we show the utility of $$\hbox {GNN}_{\mathrm{P}}$$ GNN P on SARS-Cov-2 protein targets by screening a large compound library and comparing the prediction with the experimentally measured data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Unified Language Frontend for Physic-Informed AI/ML

Artificial intelligence and machine learning (AI/ML) are becoming important tools for scientific modeling and simulation as in several other fields such as image analysis and natural language processing. ML techniques can leverage the computing power available in modern systems and reduce the human effort needed to configure experiments, interpret and visualize results, draw conclusions from huge quantities of raw data, and build surrogates for physics based models. Domain scientists in fields like fluid dynamics, microelectronics and chemistry can automate many of their most difficult and repetitive tasks or improve the design times by use of the faster ML-surrogates. However, modern ML and traditional scientific highperformance computing (HPC) tend to use completely different software ecosystems. While ML frameworks like PyTorch and TensorFlow provide Python APIs, most HPC applications and libraries are written in C++. Direct interoperability between the two languages is possible but is tedious and error-prone. In this work, we show that a compiler-based approach can bridge the gap between ML frameworks and scientific software with less developer effort and better efficiency. We use the MLIR (multi-level intermediate representation) ecosystem to compile a pre-trained convolutional neural network (CNN) in PyTorch to freestanding C++ source code in the Kokkos programming model. Kokkos is a programming model widely used in HPC to write portable, shared-memory parallel code that can natively target a variety of CPU and GPU architectures. Our compiler-generated source code can be directly integrated into any Kokkosbased application with no dependencies on Python or cross-language interfaces.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Characterization and identification of HPC applications at leadership computing facility

High Performance Computing (HPC) is an important method for scientific discovery via large-scale simulation, data analysis, or artificial intelligence. Leadership-class supercomputers are expensive, but essential to run large HPC applications. The Petascale era of supercomputers began in 2008, with the first machines achieving performance in excess of one petaflops, and with the advent of new supercomputers in 2021 (e.g., Aurora, Frontier), the Exascale era will soon begin. However, the high theoretical computing capability (i.e., peak FLOPS) of a machine is not the only meaningful target when designing a supercomputer, as the resources demand of applications varies. A deep understanding of the characterization of applications that run on a leadership supercomputer is one of the most important ways for planning its design, development and operation. In order to improve our understanding of HPC applications, user demands and resource usage characteristics, we perform correlative analysis of various logs for different subsystems of a leadership supercomputer. This analysis reveals surprising, sometimes counter-intuitive patterns, which, in some cases, conflicts with existing assumptions, and have important implications for future system designs as well as supercomputer operations. For example, our analysis shows that while the applications spend significant time on MPI, most applications spend very little time on file I/O. Combined analysis of hardware event logs and task failure logs show that the probability of a hardware FATAL event causing task failure is low. Combined analysis of control system logs and file I/O logs reveals that pure POSIX I/O is used more widely than higher level parallel I/O. Based on holistic insights of the application gained through combined and co-analysis of multiple logs from different perspectives and general intuition, we engineer features to "fingerprint" HPC applications. We use t-SNE (a machine learning technique for dimensionality reduction) to validate the explainability of our features and finally train machine learning models to identify HPC applications or group those with similar characteristic. To the best of our knowledge, this is the first work that combines logs on file I/O, computing, and inter-node communication for insightful analysis of HPC applications in production.

Liu, Zhengchun↗