Search NASA⌕ Search

SEARCH · Search NASA

Results for “scalable computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,567 records · Page 87

Tachyon: Intelligent Multi-Scale Modeling of Distributed Resilient Infrastructure and Workflows for Data Intensive HEP Analyses

The DOE High Energy Physics (HEP) program in Neutrino and Collider science drives data-intensive science and simulation on extreme-scale platforms. Modeling and optimizing the complex distributed components from experimental to leadership computing facilities are essential for HEP workflows to achieve required response times and resilience under various conditions. Tachyon proposes a framework for scalable modeling, simulation, and validation of key performance characteristics for the distributed infrastructure between FNAL and ALCF, along with associated HEP workflows.

Carothers, Chris [Rensselaer Poly.]↗

Tachyon: Intelligent Multi-Scale Modeling of Distributed Resilient Infrastructure and Workflows for Data Intensive HEP Analyses

The DOE High Energy Physics (HEP) program in Neutrino and Collider science drives data-intensive science and simulation on extreme-scale platforms. Modeling and optimizing the complex distributed components from experimental to leadership computing facilities are essential for HEP workflows to achieve required response times and resilience under various conditions. Tachyon proposes a framework for scalable modeling, simulation, and validation of key performance characteristics for the distributed infrastructure between FNAL and ALCF, along with associated HEP workflows.

Carothers, Chris [Rensselaer Poly.]↗

Detecting and Blocking Network Attacks at Ultra High Speeds

Stateful, in-depth, in-line traffic analysis for intrusion detection and prevention has grown increasingly more difficult as the data rates of modern networks rise. One point in the design space for high-performance network analysis - pursued by a number of commercial products - is the use of sophisticated custom hardware. For very high-speed processing, such systems often cast the entire analysis process in ASICs. This project pursued a different architectural approach, which we term Shunting. Shunting marries a conceptually quite simple hardware device with an Intrusion Prevention System (IPS) running on commodity PC hardware. The overall design goal is was to keep the hardware both cheap and readily scalable to future higher speeds, yet also retain the unparalleled flexibility that running the main IPS analysis in a full general-computing environment provides. The Shunting architecture we developed uses a simple in-line hardware element that maintains several large state tables indexed by packet header fields, including IP/TCP flags, source and destination IP addresses, and connection tuples. The tables yield decision values the element makes on a packet-by-packet basis: forward the packet, drop it, or divert ('shunt') it through the IPS (the default). By manipulating table entries, the IPS can, on a fine-grained basis: (i) specify the traffic it wishes to examine, (ii) directly block malicious traffic, and (iii) 'cut through' traffic streams once it has had an opportunity to 'vet' them, or (iv) skip over large items within a stream before proceeding to further analyze it. For the Shunting architecture to yield benefits, it needs to operate in an environment for which the monitored network traffic has the property that - after proper vetting - much of it can be safely skipped. This property does not universally hold. For example, if a bank needs to examine all Web traffic involving its servers for regulatory compliance, then a monitor in front of one of the bank's server farms cannot safely omit a subset of the traffic from analysis. In this environment, Shunting cannot realize its main performance benefits, and the monitoring task likely calls for using custom hardware instead. However, in many other environments we find Shunting holds promise for delivering major performance gains. This arises due to the the widely documented 'heavy tail' nature of most forms of network traffic, which we might express as 'a few of the connections carry just about all the bytes.' The key additional insight is '... and very often for these few large connections, the very beginning of the connection contains nearly all the information of interest from a security analysis perspective.' We argue that this second claim holds because it is at the beginning of connections that authentication exchanges occur, data or file names and types are specified, request and reply status codes conveyed, and encryption is negotiated. Once these occur, we have seen most of the interesting facets of the dialog. Certainly the remainder of the connection might also yield some grist for analysis, but this is generally less likely, and thus if we want to lower analysis load at as small a loss as possible of information relevant to security analysis, we might best do so by skipping the bulk of large connections. In a different context, the 'Time Machine' work by Kornexl and colleagues likewise shows that in some environments we can realize major reductions in the volume of network traffic processed, by limiting the processing to the first 10-20 KB of each connection. As a concrete example, consider an IPS that monitors SSH traffic. When a new SSH connection arrives and the Shunt fails to find an entry for it in any of its tables (per-address, per-port, per-connection), it executes the default action of diverting the connection through the IPS. The IPS analyzes the beginning of the connection in this fashion. As long as it is satisified with the dialog, it reinjects the packets forwarded to it so that the connection can continue. If the connection successfully negotiates encryption, the IPS can no longer profitably analyze it, so it downloads a per-connection table entry to the Shunt specifying that the action for the connection in the future is 'forward.' For heavy-tailed connections, this means a very large majority of the connection's packets will now pass through the Shunt device without burdening the IPS with any further analysis load. On the other hand, if the IPS is dissatisfied with some element of the initial dialog, it downloads a 'drop' entry to terminate the connection. Note that by providing for reinjection, we can promote an intrusion detection system into an intrusion prevention system, one that does not merely detect attacks but can block them before they complete. Reinjection also allows the IPS to normalize traffic to remove ambiguities that attackers can leverage to evade the IPS.

99 GENERAL AND MISCELLANEOUS↗

Measurements of three-flavor neutrino oscillations from a PISCES two-detector fit to the NOvA Experiment data

NOvA is a long-baseline neutrino oscillation experiment with two functionally identical detectors: a Near Detector (ND) at Fermilab, placed 1 km from the neutrino source, and a Far Detector (FD) located 810 km away from the ND in Minnesota. NOvA s primary physics goals are to measure the neutrino oscillation parameters $\theta_{23}$ and $\Delta m^2_{32}$ with high precision, determine the neutrino mass hierarchy, and constrain the value of $\delta_{CP}$, primarily via the study of muon neutrino to electron neutrino oscillation. Extracting values for oscillation parameters from fits to data usually relies on treating systematic uncertainties as nuisance parameters, a strategy that suffers from poor scalability as the number of uncertainties becomes larger. This work introduces PISCES (Parameter Inference with Systematic Covariance and Exact Statistics), a novel method that circumvents this scalability problem by encoding systematic uncertainties into a covariance matrix. PISCES utilizes a nested minimization in which optimal systematic pulls are first computed using the covariance matrix in an inner minimization step, then the oscillation parameters are profiled over in the outer minimization. PISCES also uses a Poisson Likelihood term, making it ideal for the inclusion of low-statistic samples in the fits. PISCES is a flexible framework that also supports complex fits, such as a joint Near and Far detector fit. In the standard NOvA analysis, oscillation parameters are extracted using an extrapolation technique in which the ND data indirectly constrain the FD prediction via a ratio method. PISCES, on the other hand, enables a simultaneous ND+FD fit, allowing the high-statistics ND data to directly constrain systematic uncertainties across all samples. This thesis presents the full PISCES joint ND+FD fit for the NOvA three-flavor analysis, details its implementation, and evaluates its performance through extensive robustness tests and fake data studies. It also provides a comparison between the PISCES joint ND+FD results and the standard NOvA extrapolation method using the full NOvA 10-year data set. The results demonstrate that PISCES can successfully fit NOvA data while incorporating the constraints from the ND detectors consistently, using physically motivated systematic uncertainties to account for data/MC discrepancies.

Rajaoalisoa, Miriama [Cincinnati U.]↗

FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica

Numerical simulations of Greenland and Antarctic ice sheets involve the solution of large-scale highly nonlinear systems of equations on complex shallow geometries. This work is concerned with the construction of Schwarz preconditioners for the solution of the associated tangent problems, which are challenging for solvers mainly because of the strong anisotropy of the meshes and wildly changing boundary conditions that can lead to poorly constrained problems on large portions of the domain. Here, two-level GDSW (Generalized Dryja–Smith–Widlund) type Schwarz preconditioners are applied to different land ice problems, i.e., a velocity problem, a temperature problem, as well as the coupling of the former two problems. We employ the MPI-parallel implementation of multi-level Schwarz preconditioners provided by the package FROSch (Fast and Robust Schwarz)from the Trilinos library. The strength of the proposed preconditioner is that it yields out-of-the-box scalable and robust preconditioners for the single physics problems. To our knowledge, this is the first time two-level Schwarz preconditioners are applied to the ice sheet problem and a scalable preconditioner has been used for the coupled problem. The pre-conditioner for the coupled problem differs from previous monolithic GDSW preconditioners in the sense that decoupled extension operators are used to compute the values in the interior of the sub-domains. Several approaches for improving the performance, such as reuse strategies and shared memory OpenMP parallelization, are explored as well. In our numerical study we target both uniform meshes of varying resolution for the Antarctic ice sheet as well as non uniform meshes for the Greenland ice sheet are considered. We present several weak and strong scaling studies confirming the robustness of the approach and the parallel scalability of the FROSch implementation. Among the highlights of the numerical results are a weak scaling study for up to 32 K processor cores (8 K MPI-ranks and 4 OpenMP threads) and 566 M degrees of freedom for the velocity problem as well as a strong scaling study for up to 4 K processor cores (and MPI-ranks) and 68 M degrees of freedom for the coupled problem.

58 GEOSCIENCES↗

UNITY: Unified Memory and File Space. In: ROSS '17 Proceedings of the 7th International Workshop on Runtime and Operating Systems for Supercomputers ROSS 2017, Article No. 6

This paper describes the vision for UNITY, a new high-performance computing focused data storage abstraction that places the entire memory hierarchy, including both traditionally separated memory-and file-based data storage, into one storage continuum. Through the use of a novel API and a set of services centered around a smart runtime system, UNITY is able to provide a number of valuable and interesting benefits. The unified storage space provides a scalable and resilient data environment that dynamically manages the mapping of data onto available resources based on multiple factors, including desired persistence and energy budget considerations. By eliminating the need for high-performance computing domain scientists to develop architecture-dependent optimizations for rapidly evolving data storage technologies, UNITY addresses both ease-of-use and performance.

Jones, Terry↗

MetricForensics: A Multi-Level Approach for Mining Volatile Graphs

Advances in data collection and storage capacity have made it increasingly possible to collect highly volatile graph data for analysis. Existing graph analysis techniques are not appropriate for such data, especially in cases where streaming or near-real-time results are required. An example that has drawn significant research interest is the cyber-security domain, where internet communication traces are collected and real-time discovery of events, behaviors, patterns and anomalies is desired. We propose MetricForensics, a scalable framework for analysis of volatile graphs. MetricForensics combines a multi-level “drill down" approach, a collection of user-selected graph metrics and a collection of analysis techniques. At each successive level, more sophisticated metrics are computed and the graph is viewed at a finer temporal resolution. In this way, MetricForensics scales to highly volatile graphs by only allocating resources for computationally expensive analysis when an interesting event is discovered at a coarser resolution first. We test MetricForensics on three real-world graphs: an enterprise IP trace, a trace of legitimate and malicious network traffic from a research institution, and the MIT Reality Mining proximity sensor data. Our largest graph has »3M vertices and »32M edges, spanning 4:5 days. The results demonstrate the scalability and capability of MetricForensics in analyzing volatile graphs; and highlight four novel phenomena in such graphs: elbows, broken correlations, prolonged spikes, and strange stars.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Order parameter re-mapping algorithm for 3D phase field model of grain growth using FEM

Phase field modeling (PFM) is a well-known technique for simulating microstructural evolution. To model grain growth using PFM, typically each grain is assigned a unique non-conserved order parameter and each order parameter field is evolved in time. Traditional approaches using a one-to-one mapping of grains to order parameters present a challenge when modeling large numbers of grains due to the computational expense of using many order parameters. This problem is exacerbated when using an implicit finite element method (FEM), as the global matrix size is proportional to the number of order parameters. While previous work has developed methods to reduce the number of required variables and thus computational complexity and run time, none of the existing approaches can be applied for an implicit FEM implementation of PFM. Here, we present a modular, dynamic, scalable reassignment algorithm suitable for use in such a system. Polycrystal modeling with grain growth and stress require careful tracking of each grain’s position and orientation which is lost when using a reduced order parameter set. In conclusion, the method presented in this paper maintains a unique ID for each grain even after reassignment, to allow the PFM to be tightly coupled to calculations of the stress throughout the polycrystal. Implementation details and comparative results of our approach are presented.

36 MATERIALS SCIENCE↗

Distributed memory parallel Markov random fields using graph partitioning

Markov random fields (MRF) based algorithms have attracted a large amount of interest in image analysis due to their ability to exploit contextual information about data. Image data generated by experimental facilities, though, continues to grow larger and more complex, making it more difficult to analyze in a reasonable amount of time. Applying image processing algorithms to large datasets requires alternative approaches to circumvent performance problems. Aiming to provide scientists with a new tool to recover valuable information from such datasets, we developed a general purpose distributed memory parallel MRF-based image analysis framework (MPI-PMRF). MPI-PMRF overcomes performance and memory limitations by distributing data and computations across processors. The proposed approach was successfully tested with synthetic and experimental datasets. Additionally, the performance of the MPI-PMRF framework is analyzed through a detailed scalability study. We show that a performance increase is obtained while maintaining an accuracy of the segmentation results higher than 98%. The contributions of this paper are: (a) development of a distributed memory MRF framework; (b) measurement of the performance increase of the proposed approach; (c) verification of segmentation accuracy in both synthetic and experimental, real-world datasets

97 MATHEMATICS AND COMPUTING↗

UCX: An Open Source Framework for HPC Network APIs and Beyond, In: 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects

This paper presents Unified Communication X (UCX), a set of network APIs and their implementations for high throughput computing. UCX comes from the combined effort of national laboratories, industry, and academia to design and implement a high-performing and highly-scalable network stack for next generation applications and systems. UCX design provides the ability to tailor its APIs and network functionality to suit a wide variety of application domains and hardware. We envision these APIs to satisfy the networking needs of many programming models such as Message Passing Interface (MPI), OpenSHMEM, Partitioned Global Address Space (PGAS) languages, task-based paradigms and I/O bound applications. To evaluate the design we implement the APIs and protocols, and measure the performance of overhead-critical network primitives fundamental for implementing many parallel programming models and system libraries. Our results show that the latency, bandwidth, and message rate achieved by the portable UCX prototype is very close to that of the underlying driver. With UCX, we achieved a message exchange latency of 0.89 us, a bandwidth of 6138.5 MB/s, and a message rate of 14 million messages per second. As far as we know, this is the highest bandwidth and message rate achieved by any network stack (publicly known) on this hardware.

Computer Science↗

Automatic generation of executable communication specifications from parallel applications

Portable parallel benchmarks are widely used and highly effective for (a) the evaluation, analysis and procurement of high-performance computing (HPC) systems and (b) quantifying the potential benefits of porting applications for new hardware platforms. Yet, past techniques to synthetically parameterized hand-coded HPC benchmarks prove insufficient for today's rapidly-evolving scientific codes particularly when subject to multi-scale science modeling or when utilizing domain-specific libraries. To address these problems, this work contributes novel methods to automatically generate highly portable and customizable communication benchmarks from HPC applications. We utilize ScalaTrace, a lossless, yet scalable, parallel application tracing framework to collect selected aspects of the run-time behavior of HPC applications, including communication operations and execution time, while abstracting away the details of the computation proper. We subsequently generate benchmarks with identical run-time behavior from the collected traces. A unique feature of our approach is that we generate benchmarks in CONCEPTUAL, a domain-specific language that enables the expression of sophisticated communication patterns using a rich and easily understandable grammar yet compiles to ordinary C + MPI. Experimental results demonstrate that the generated benchmarks are able to preserve the run-time behavior - including both the communication pattern and the execution time - of the original applications. Such automated benchmark generation is particularly valuable for proprietary, export-controlled, or classified application codes: when supplied to a third party. Our auto-generated benchmarks ensure performance fidelity but without the risks associated with releasing the original code. This ability to automatically generate performance-accurate benchmarks from parallel applications is novel and without any precedence, to our knowledge.

42 ENGINEERING↗

Scalable line and plane relaxation in a parallel structured multigrid solver

The efficient solution of sparse, linear systems that arise through the discretization of partial differential equations remains a key challenge for a range of high performance scientific simulations. One approach for reducing data movement and improving performance is by exposing and exploiting structure in a problem through the use of robust structured multilevel solvers. By choosing coarsening that preserves the structure of the problem, these methods maintain efficient structured computation and communication throughout the multigrid hierarchy. However, when coarsening is not permitted to be dependent on the operator, anisotropy must be addressed by the smoother — producing error compatible for coarse-grid correction with structured coarsening. Here, the components required in a scalable parallel structured solver are described with a focus on memory and communication efficiency of robust smoothers. While the implementation of communication and memory reduction techniques in smoothers integrated in a complete 3D solver present a significant engineering challenge, a novel approach is proposed that addresses these challenges systematically through a change to the solver’s execution model. Enabled by user-level threading paired with a set of data and communication abstractions, this approach permits seamless aggregation of communication in plane smoothers — directly reusing code for a 2D distributed multilevel cycle. Results show an effective reduction in communication costs for coarse-grid problems, and result in a speedup of 8.7x in smoothing routines shown in Fig. 12 using this approach. This produces a significant improvement to strong scalability while maintaining favorable weak scaling behavior. Finally, a parallel scaling study using a series of refined meshes is included that demonstrates the effectiveness of this approach in an application of interest.

97 MATHEMATICS AND COMPUTING↗

Quantum learning advantage on a scalable photonic platform

Recent advances in quantum technologies have demonstrated that quantum systems can outperform classical ones in specific tasks, a concept known as quantum advantage. Although previous efforts have focused on computational speedups, a definitive and provable quantum advantage that is unattainable by any classical system has remained elusive. Here, in this work, we demonstrate a provable photonic quantum advantage by implementing a quantum-enhanced protocol for learning a high-dimensional physical process. Using imperfect Einstein–Podolsky–Rosen entanglement, we achieve a sample complexity reduction of 11.8 orders of magnitude compared to classical methods without entanglement. These results show that large-scale, provable quantum advantage is achievable with current photonic technology and represent a key step toward practical quantum-enhanced learning protocols in quantum metrology and machine learning.

Liu, Zheng-Hao [Technical Univ. of Denmark, Lyngby↗

Quantum Computing in Next-Generation Transportation Optimization

We explore how quantum computing (QC) can advance transportation optimization, with a focus on two high-impact areas: traffic signal control and vehicle electrification with grid integration. As transportation systems grow in complexity, classical optimization methods increasingly struggle to deliver scalable and efficient solutions, particularly for real-time, data-rich environments. This work identifies key challenges within these two domains where QC may offer advantages, particularly in handling combinatorial decision spaces and dynamic constraints. We begin by outlining the limitations of classical approaches for traffic signal control optimization and electric vehicle charging coordination, highlighting where computational limitations arise. Previous quantum formulations are presented and new formulations are proposed to demonstrate how emerging quantum algorithms, including quantum annealing and the Quantum Approximation Optimization Algorithm, could be leveraged to reformulate and address these problems. We also evaluate the suitability of current quantum hardware and discuss recent trends that indicate when QC may become a viable tool for transportation applications. While acknowledging the present limitations of QC technologies, this poster emphasizes the importance of preparing quantum-compatible models today. By reviewing and establishing formulations that align with the strengths of quantum algorithms, researchers and practitioners can better position themselves to take advantage of QC advancements as they occur. This work aims to provide a practical, forward-looking perspective on the near-term potential of quantum computing in transportation optimization.

33 ADVANCED PROPULSION SYSTEMS↗

Machine Learning for Scalable and Optimal Load Shedding Under Power System Contingency

Prompt and effective corrective actions in response to unexpected contingencies are crucial for improving power system resilience and preventing cascading blackouts. The optimal load shedding (OLS) accounting for network limits has the potential to address the diverse system-wide impacts of contingency scenarios as compared to traditional local schemes. However, due to the fast cascading propagation of initial contingencies, real-time OLS solutions are challenging to attain in large systems with high computation and communication needs. In this paper, we propose a decentralized design that leverages offline training of a neural network (NN) model for individual load centers to autonomously construct the OLS solutions from locally available measurements. Our learning-for-OLS approach can greatly reduce the computation and communication needs during online emergency responses, thus preventing the cascading propagation of contingencies for enhanced power grid resilience. Numerical studies on both the IEEE 118-bus system and a synthetic Texas 2000-bus system have demonstrated the efficiency and effectiveness of our scalable OLS learning design for timely power system emergency operations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Scenario generation for stochastic optimization problems via the sparse grid method

We study the use of sparse grids in the scenario generation (or discretization) problem in stochastic programming problems where the uncertainty is modeled using a continuous multivariate distribution. We show that, under a regularity assumption on the random function involved, the sequence of optimal objective function values of the sparse grid approximations converges to the true optimal objective function values as the number of scenarios increases. The rate of convergence is also established. We treat separately the special case when the underlying distribution is an affine transform of a product of univariate distributions, and show how the sparse grid method can be adapted to the distribution by the use of quadrature formulas tailored to the distribution. We numerically compare the performance of the sparse grid method using different quadrature rules with classic quasi-Monte Carlo (QMC) methods, optimal rank-one lattice rules, and Monte Carlo (MC) scenario generation, using a series of utility maximization problems with up to 160 random variables. The results show that the sparse grid method is very efficient, especially if the integrand is sufficiently smooth. In such problems the sparse grid scenario generation method is found to need several orders of magnitude fewer scenarios than MC and QMC scenario generation to achieve the same accuracy. As a result, it is indicated that the method scales well with the dimension of the distribution--especially when the underlying distribution is an affine transform of a product of univariate distributions, in which case the method appears scalable to thousands of random variables.

97 MATHEMATICS AND COMPUTING↗

Optimization of CPAPR for x64 multicore

I report the progress to date of my work on scaling the CPAPR algorithm and necessary supporting code to enable processing large (gigabyte to 100 gigabyte) data sets and benchmarking the same. Where possible, I also report background information possibly of relevance in future modifications of the code. The results include: minor repairs and additions to the TTB library for portability, algorithmic improvements relevant to both serial and multithreaded implementations, algorithmic improvements taking advantage of multithreading hardware, support library additions (binary IO routines) needed for efficiently and reproducibly benchmarking the algorithms. For this optimization work, no large scale data sets are available. Therefore, scalability of data synthesis algorithms is addressed as well.

97 MATHEMATICS AND COMPUTING↗

Alliance Project: Secure Control Systems for the Energy Sector

The Alliance Project addresses Area of Interest 4: Secure Remote Access for the Energy Sector, by combining physical and cyber access under one cybersecurity access system. The Alliance Project built on the successful completion of the Lemnos and Padlock Project (DOE-Funded Project started in 2007 and 2010) and researched and developed a solution for an Ethernet Security Gateway with the functionality required to integrate the physical access control cyber system and interface with legacy and new proximity card readers used for physical access requests. This project focused on securing and enhancing the cyber system the physical access control system operates on. The team tested and demonstrated the system solution on the SEL campus. Currently the physical access cyber system is completely separate from the cyber access control cyber system. With a single system industry benefits include: (1) Single system to learn and teach employees; (2) Single system that collects and alerts on logs and events simplifying incident response plans and NERC CIP compliance; (3) Unifies trust management to a single system reducing the potential errors in privilege escalation or revocation; (4) Provides a stronger cybersecurity platform the physical access control system operates on. Today they are mostly plain text protocols and little cyber awareness; (5) Integration of physical access awareness allows the cyber system to react and adjust. In some cases, the cyber access can be enhanced with authorized physical access and other cases the cyber access can be more restricted with unauthorized physical access; and (6) Enables the utility to establish as large or as small a physical access control system as they require. One example would be when a utility shares a substation with a neighbor utility and can’t rely on the substation door for the physical access control. The Alliance technology enables them to lock equipment racks simply with no scalability issues because it stands on the cyber security infrastructure which is already highly scalable.

24 POWER TRANSMISSION AND DISTRIBUTION↗