SEARCH · Search NASA

Results for “data intensive computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

DOE OSTI2019

Xavier: High-Performance X-Drop Adaptive Banded Pairwise Alignment (Xavier) v1.0

Pairwise sequence alignment is one of the most compute-intense challenges for the analysis of genomics data and it can account for more than 90% of the runtime depending on the application. Despite the effort to reduce the computational cost introducing heuristics such as banded and x-drop alignment, the performance of state-of-the-art software is often unsatisfactory for long sequences. Recent advances in single-molecule sequencing technologies keep increasing the length of the sequences to align at the cost of higher error rates b ranging from 5% to 15% b, emphasizing the need for fast and accurate software. Our long-read alignment software, named Xavier, computes high-performance x-drop adaptive banded pairwise alignment exploiting instruction-level parallelism via Single-Instruction-Multiple-Data (SIMD) operations. Xavier uses a narrow bandwidth that appreciably improves performance reducing the search space for the optimal alignment. In our experiment, the adaptive mechanism enables Xavier to find the optimal alignment between related sequences even when the error rate is as high as 50%. Xavier achieves speed-up up to 5X over competitor software when enabling the x-drop termination and up to 415X when disabling it, with negligible accuracy loss.

Guidi, Giulia↗

DOE OSTI2016

Further Investigation of Employing Binomial Small-Sample Estimator in M-LROM

Reduced Order Modeling (ROM) is essential to enable the identification of the so-called active parameter subspace for high-fidelity models that are employed in computationally intensive analyses such as uncertainty characterization and data assimilation. Given the complexity of models for real-world application, brute force methods to identify the active subspace may not be computationally efficient to realize the value of reduction, since in realistic problems, the full scale model must be executed few hundred times, which may be computationally taxing. Earlier work has introduced a new approach, which represents a further development of our previous work on ROM, referred to as multi-level ROM (M-LROM). In M-LROM, approximate models are employed to identify the active subspace instead of the full scale model. In this summary, we present our recent results on M-LROM, specifically in regard to the construction of reliable error metrics that can bound the errors resulting from the reduction. The idea is to employ a lower fidelity model, i.e., an approximation of the full scale or high fidelity model, to build the active subspace, and via the proposed error bound, the quality of the active subspace can be determined by calculating an upper bound on the errors resulting from the reduction. For this approach to be practical, the error bound must be realistic, implying that it should be close to the actual errors expected, and not much higher. Therefore, an important goal of this summary is to compare different error bounds in terms of the relation between their estimated bounds to the actual errors. In particular, we compare the proposed error bound which numerically construct an estimator using binomial distribution to the widely used small-sample norm estimators. Unlike most probabilistic error estimation methods, the proposed bound employs a binomial distribution to construct the estimator whereas in other conventional methods, the distribution employed is typically the standard Gaussian, normalized over a unit hypersphere as initially proposed by the seminal paper of J. Dixon on random matrix theory. Previous publications provided evidence that the binomial distribution calculates more realistic bounds as compared to the Gaussian distribution, which tends to be conservative, implying that the bound is much bigger than the actual errors, which diminishes the value of the reduction, as now must increase the size of the active subspace to reach the desired upper bound on the reduction errors. This work has provided a rigorous approach by which error bounds on the reduction errors can be established without having to execute the high fidelity model during the construction of the active subspace. The high fidelity model is only need to test the error bound using small number of executions. This methodology paves the way for a more efficient application of ROM to complex engineering models, where standard ROM techniques may be too taxing to optimally realize their value in practical situations. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2022

The LLNL nuclear data infrastructure for the GNDS data format

The next generation of nuclear data infrastructure tools at the Livermore National Laboratory (LLNL) consists of pipeline of codes that read and process nuclear data from evaluated files saved in the new GNDS (Generalised Nuclear Data Structure) nuclear data format. The processing code FUDGE (For Updating Data and Generating Evaluations) is at the front-end of this pipeline as it reads and process the evaluated data for use in downstream transport codes. FUDGE is Python based with C and C++ extensions for computationally intensive tasks. As is the case for the evaluated data, the processed output is also saved in the GNDS format and the GIDI+ API is provided as the interface between the processed data and the transport codes. GIDI+ is a C++ based suite of codes and it includes GIDI (General Interaction Data Interface), a library for reading and writing GNDS data, and MCGIDI which is the cross section lookup, and reaction and product distribution sampling interface between Monte Carlo transport codes and the GNDS data. GIDI provides methods for easy access to the multi-group processed GNDS data and this is demonstrated through its implementation in ARDRA, the LLNL deterministic transport code. The evaluation and sampling methods in MCGIDI are available as both CPU and GPU methods which facilitates the use of MCGIDI in both traditional CPU-based as well as the next generation mixed model computational architectures. This is demonstrated through the GIDI+ implementation in MERCURY, the LLNL Monte Carlo transport code. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2020

Methods and Experiences for Developing Abstractions for Data-intensive, Scientific Applications

Developing software for scientific applications that require the integration of diverse types of computing, instruments, and data present challenges that are distinct from commercial software. These applications require scale, and the need to integrate various programming and computational models with evolving and heterogeneous infrastructure. Pervasive and effective abstractions for distributed infrastructures are thus critical; however, the process of developing abstractions for scientific applications and infrastructures is not well understood. While theory-based approaches for system development are suited for well-defined, closed environments, they have severe limitations for designing abstractions for scientific systems and applications. The design science research (DSR) method provides the basis for designing practical systems that can handle real-world complexities at all levels. In contrast to theory-centric approaches, DSR emphasizes both practical relevance and knowledge creation by building and rigorously evaluating all artifacts. In this work, we show how DSR provides a well-defined framework for developing abstractions and middleware systems for distributed systems. Specifically, we address the critical problem of distributed resource management on heterogeneous infrastructure over a dynamic range of scales, a challenge that currently limits many scientific applications. We use the pilot-abstraction, a widely used resource management abstraction for high-performance, high throughput, big data, and streaming applications, as a case study for evaluating the DSR activities. For this purpose, we analyze the research process and artifacts produced during the design and evaluation of the pilot-abstraction. We find DSR provides a concise framework for iteratively designing and evaluating systems. Finally, we capture our experiences and formulate different lessons learned.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Studying CPU and memory utilization of applications on Fujitsu A64FX and Nvidia Grace Superchip

ARM-based manycore CPU architectures are well-positioned to provide the rising memory throughput requirements of modern data intensive scientific applications in High Performance Computing (HPC). The Fujitsu A64FX CPU platform is based on the ARM v8.2A architecture, and is the processor of the flagship Japanese supercomputer - "Fugaku", which was previously ranked as the #1 supercomputer in the world according to the Top500 list. The Nvidia Grace superchip features 144 Neoverse V2 cores based on the ARMv9 architecture with 4x128b SVE2, providing exceptional computational power. The chip supports up to 480GB of memory, making it ideal for AI, machine learning, and scientific computing workloads. In this paper, we conduct a thorough performance exploration of a variety of parallel bandwidth-sensitive benchmarks and applications compiled with the native Fujitsu compiler on a Fugaku A64FX compute node and ARM (LLVM) Compiler on an NVIDIA Grace superchip compute node, engaging all the computational cores per cluster using OpenMP multithreading (assuming the cores can drive the available bandwidth). Our ultimate goals are to study the resource utilization of scientific applications and benchmarks on A64FX and Grace superchip, considering graph application scenarios ( GAP Benchmark suite) and eleven appli- cation proxies from the Rodinia heterogeneous benchmark suite (considering domains such as Data Mining, Bioinformatics, Fluid Dynamics, Pattern Recognition, etc.). Through exhaustive performance monitoring, we quantify the resource utilization of diverse OpenMP-based HPC applications on both the Fujitsu A64FX and the Nvidia Grace Superchip platforms.

benchmarking, Performance Analysis, High performan↗

DOE OSTI2022

Deep learning multiphysics network for imaging CO 2 saturation and estimating uncertainty in geological carbon storage

Multiphysics inversion exploits different types of geophysical data that often complement each other and aims to improve overall imaging resolution and reduce uncertainties in geophysical interpretation. Despite the advantages, traditional multiphysics inversion is challenging because it requires a large amount of computational time and intensive human interactions for preprocessing data and finding trade-off parameters. These issues make it nearly impossible for traditional multiphysics inversion to be applied as a real-time monitoring tool for geological carbon storage. In this paper, we present a deep learning (DL) multiphysics network for imaging CO 2 saturation in real time. The multiphysics network consists of three encoders for analysing seismic, electromagnetic and gravity data and shares one decoder for combining imaging capabilities of the different geophysical data for better predicting CO 2 saturation. The network is trained on pairs of CO 2 label models and multiphysics data so that it can directly image CO 2 saturation. Here we use the bootstrap aggregating method to enhance the imaging accuracy and estimate uncertainties associated with CO 2 saturation images. Using realistic CO 2 label models and multiphysics data derived from the Kimberlina CO 2 storage model, we evaluate the performance of the deep learning multiphysics network and compare its imaging results to those from the deep learning single-physics networks. Our modelling experiments show that the deep learning multiphysics network for seismic, electromagnetic, and gravity data not only improves the imaging accuracy but also reduces uncertainties associated with CO 2 saturation images. Our results also suggest that the deep learning multiphysics network for the non-seismic data (i.e., electromagnetic and gravity) can be used as an effective low-cost monitoring tool in between regular seismic monitoring.

58 GEOSCIENCES↗

NASA NTRSDate not supplied

Enabling Space Biological Knowledge Discovery Through Image and Video Data Sharing

Increased biomedical risks and challenges associated with deep space missions and experiments (cis-Lunar, Mars transit/surface) require new knowledge discovery and development of novel ecosystems. Supporting distant and long-duration missions and experiments requires biological data (from yeast, microbes, fruit flies, C. elegans, plants, crops, rodents, humans) be findable, accessible, interoperable, reusable (FAIR), and maximally open-access. As data-intensive, bioinformatic, meta-analytical, and computer-assisted approaches continue to be a centerpiece of modern research, the NASA Biological and Physical Sciences division is expanding its Open Science capabilities beyond NASA GeneLab. The NASA Ames Life Sciences Data Archive (ALSDA) is a repository which is responsible for collecting and access to space biological imagery and video, alongside tabular and environmental data. In this presentation, we will discuss strategies dealing with archiving, curating, and accessibility of images from very distinct imaging modalities (e.g., micro-computed tomography, magnetic resonance imaging, photographic images of plants, fluorescence microscopy, behavioral videos, etc.). There are two main challenges: 1. Open-source data storage and 2. Metadata related to the imagery-video. Both have been solved by leveraging two existing open-source systems. For data storage, ALSDA is utilizing components through the Open Microscopy Environment (OME), which can read most imaging proprietary formats and display on a web interface complex multidimensional images (Z stack, multi-channel, temporal, spectral). Most technical metadata from imaging modalities are captured seamlessly. For metadata capturing experimental details, ALSDA (like GeneLab) uses the ISA-Tab specification which relies on the ISA data model to order and classify metadata. The ISA data model uses a tree structure with three files to capture the metadata: The top layer is the Investigations file, the second layer is the Study file(s), and the last layer is the Assay file(s). We believe such an approach may be useful for other types of image research data from other investigators in the AGU community.

imaging↗

NASA NTRSDate not supplied

Using Coordinated, Multi-Agent Platforms for Dynamic Ocean Worlds Science

Planetary science missions have the opportunity to enhance science return through deployment of autonomous capabilities designed to dynamically respond to new information. Future outer solar system missions to ocean worlds in particular would benefit from this technology - intelligent science payloads (ISP) - because it would allow for a coordinated, near real-time response to ephemeral ‘events’ such as plumes, tectonism, surface implantation, volatile releases, thermal and magnetic anomalies, or radiation, as well as increasing the cadence and coverage of data collection. Prioritization and decision-making frameworks from ISP could be deployed at various scales - from analysis onboard a spacecraft with multiple instruments – to coordinated analyses among separate spacecraft in an e.g., distributed systems mission (DSM) composed of multiple SmallSats. Goddard’s Intelligent Science Payload team is developing an agile autonomous architecture for an icy ocean worlds DSM concept. Our goals are to coordinate data collection and onboard data analysis, and to make autonomous decisions for new data collection and analysis based on science priorities between multiple spacecraft with variable instrumentation and orbits. We use a range of data analysis tools to coordinate the DSM response, spanning from observations of data over a specified threshold to more computationally intensive machine learning algorithms (ML). ML algorithms here currently focus on determining the composition of an ocean world using mass spectrometry, and specifically methods for understanding ‘novelties’ and potential biosignatures. These algorithms could be used to quickly process and analyze onboard data that would be significantly delayed in downlink due to long communication delays for outer solar system missions in order to make dynamic science observations. Our ocean worlds case study ISP architecture is intended as an ‘agile’ and modular framework that could be used as a whole or as particular modules based on mission needs.

Distributed Systems↗

DOE OSTI2022

A review of water valuation metrics: Supporting sustainable water use in manufacturing

In the manufacturing sector, water has been often considered too cheap to conserve. Such thinking relies on water valuations that limit the value of water to the price paid. Using such simple methods, the share of water cost to total manufacturing cost is significantly small, <3%. As a result, conserving water and enabling technology uptake is difficult to justify economically and slow to advance, hindering progress toward sustainable water use. However, the value of water to a manufacturer is far greater than the price paid. Valuations such as the true cost of water consider the additional in-plant treatment and energy costs and have been gaining greater traction in the manufacturing sector. However, true cost alone still undervalues water by not accounting for economic and social costs related to scarcity and environmental externalities. This paper makes the case and presents a framework for valuing manufacturing water beyond the price paid and the true cost. The proposed fuller valuation of manufacturing water takes into account the internal and opportunity costs associated with the realization of water risks. The paper follows with a review of a wide range of water valuation metrics, both at the specific industry level and regional/economy-wide level. The use of various valuation metrics incorporating the relationship between the change in value with change in water use, such as marginal value of water, shadow price, and elasticity at the specific industry level, has been limited in the U.S. manufacturing sector. Further, a limited number of studies exist on data-intensive subjective evaluation techniques such as computable general equilibrium modeling and input-output modeling for regional water valuation. After reviewing water value metrics, several recent case studies from manufacturers from the literature are presented to illustrate both the promise and challenges of a fuller valuation of water as proposed here. Some large multinational corporations have moved toward assessing the value of water via supply chain sustainability initiatives, environmental profit and loss accounting, estimating risk-adjusted values of water, hydro-economic modeling, natural capital asset valuation, and developing value chain indices. This paper provides policymakers and technology developers a framework for monetizing water value beyond its true cost and current metrics. If adopted, such fuller water valuations can help make the business case for the development and deployment of cost-effective water-conserving technologies, thereby improving the sustainability of the manufacturing sector with respect to water.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Empirically-calibrated H100 node power models for accurate AI training energy estimation

Accurately quantifying the energy use of artificial intelligence (AI) training is critical for infrastructure planning, carbon accounting, and sustainable data center operation, but few studies have directly measured the power consumption of production workloads on contemporary hardware. By combining empirical measurements from Brookhaven National Laboratory during AI training on 8-graphics-processing-unit H100 systems with open-source benchmarking data, we develop statistical models relating computational intensity to node-level power consumption. We measure the gap between manufacturer-rated thermal design power (TDP) and actual power demand during AI training. Our analysis reveals that even computationally intensive workloads operate at only 76% of the 10.2 kW TDP rating. Our architecture-specific model, calibrated to floating-point operations, predicts energy consumption with 11.4% mean absolute percentage error, significantly outperforming TDP-based approaches (27%–37% error). We identified distinct power signatures between transformer and convolutional neural network architectures, with transformers showing characteristic fluctuations that may impact grid stability. These results provide a measurement-grounded basis for improving AI training energy estimates, enabling more reliable infrastructure sizing, cost projections, and environmental impact assessments.

Newkirk, Alex C↗

DOE OSTI2024

TomoPyUI : a user-friendly tool for rapid tomography alignment and reconstruction

The management and processing of synchrotron and neutron computed tomography data can be a complex, labor-intensive and unstructured process. Users devote substantial time to both manually processing their data ( i.e. organizing data/metadata, applying image filters etc. ) and waiting for the computation of iterative alignment and reconstruction algorithms to finish. In this work, we present a solution to these problems: TomoPyUI , a user interface for the well known tomography data processing package TomoPy . This highly visual Python software package guides the user through the tomography processing pipeline from data import, preprocessing, alignment and finally to 3D volume reconstruction. The TomoPyUI systematic intermediate data and metadata storage system improves organization, and the inspection and manipulation tools (built within the application) help to avoid interrupted workflows. Notably, TomoPyUI operates entirely within a Jupyter environment. Herein, we provide a summary of these key features of TomoPyUI , along with an overview of the tomography processing pipeline, a discussion of the landscape of existing tomography processing software and the purpose of TomoPyUI , and a demonstration of its capabilities for real tomography data collected at SSRL beamline 6-2c.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2024

Community Input on the Need, Scope, and Development Roadmap of a Proposed National Virtual Biosecurity for Bioenergy Crops Center (NVBBCC)

Brookhaven National Laboratory (BNL) was awarded a pilot project in FY22 under the U.S. Department of Energy (DOE) Office of Science Biopreparedness Research Virtual Environment (BRaVE) initiative, to define research priorities, needs, and requirements for a national virtual center devoted to the biosecurity of bioenergy crops. The mission of the proposed center, referred to as the National Virtual Biosecurity for Bioenergy Crop Center (NVBBCC), would be to provide the scientific basis and tools to detect, characterize, model, and mitigate biothreats to bioenergy crops. This function will be essential to ensure the projected increased US reliance over the next few decades on key plant-based energy products, such as biojet fuel. The NVBBCC is envisioned as a distributed, virtual center with multiple national laboratories at its core to maximize the use of existing unique facilities and expertise across the DOE complex. A major goal of the pilot project was to develop a roadmap for establishing NVBBCC through a series of meetings to gather community input. A total of about 150 individuals, drawn from DOE laboratories, the USDA, academia, NIH, DHS and the private sector participated in six planning meetings held in FY23. Four of the meetings were focused on specific research topics (disease detection, dispersion and disease propagation, biomolecular characterization of plant-pathogen interaction, and mitigation strategies). These four meetings were followed by a meeting that focused on computational needs to support collaborative, data-intensive research within a distributed center as well as workforce development. A final meeting focused on establishing and maintaining preparedness within NVBBCC to respond to an emerging disease within bioenergy crops and how it would collaborate and coordinate with USDA and DHS.

09 BIOMASS FUELS↗

NASA NTRS1973

Analysis of ground tests of a microwave, earth-occultation, pressure-reference-level system.

A two-satellite, microwave, earth-occultation system can supplement an infrared sensor by providing an accurate altitude reference that will serve to fix, as a function of height, the derived temperature profile of the infrared sounder. The results of ground tests made in Hawaii to estimate the likely effects of scintillation and fading on an occultation system are described. It was found that the microwave signal suffered periods of intense fading; extensive computer analyses of the data were performed in which aircraft-generated refractivity profiles were subjected to ray tracing. Results of these analyses indicate that the probable cause of the observed fading was multipath, a low-altitude phenomenon usually attributed to water vapor inhomogeneities. It is maintained that multipath will therefore have minimal effect on the pressure-reference microwave occultation system, which would operate at a relatively high closest-approach altitude (about 8 km).

Ungar, S. G.↗

DOE OSTI2017

Partnership for Edge Physics Simulation

A major goal of our participation in the Edge Physics Simulation project has been to contribute to the understanding of the self-organization of tokamak turbulence fluctuations resulting in the formation of a staircase structure in the ion temperature. A second important goal is to demonstrate how small scale turbulence in plasmas self-organizes with dynamically driven quasi-stationary flow shear. These goals have been accomplished through the analyses of the statistical properties of XGC1 flux driven Gyrokinetic electrostatic ion temperature gradient (ITG) turbulence simulation data in which neutrals are included. The ITG turbulence data, and in particular fluctuation data, were obtained from a massively parallel flux-driven gyrokinetic full-f particle-in-cell simulation of a DIII-D like equilibrium. Below some the findings are summarized. It was observed that the emergence of staircase structure is related to the variations in the normalized temperature gradient length (R/LT) and the poloidal flow shear. Average turbulence intensity is found to be large in the vicinity of minima in R/LTi, where ITG growth is expected to be lower. The distributions of the occurrences of potential fluctuation are found to be Gaussian away from the staircase-step locations, but they are found to be non-Gaussian in the vicinity of staircase-step locations. The results of analytically derived expressions for the distribution of the occurrences of turbulence intensity and intensity flux were compared with the corresponding quantities computed using XGC1 simulation data and good agreement is found. The derived expressions predicts inward and outward propagation of turbulence intensity flux in an intermittent fashion. The outward propagation of turbulence intensity flux occurs at staircase-step locations and is related to the change in poloidal flow velocity shear and to the change in the ion temperature gradient. The standard deviation, skewness and kurtosis for turbulence quantities were computed and found to be large in the vicinity of the staircase-step structures. Large values of skewness and kurtosis can be explained by a temporary opening and closing of the structure which allows turbulence intensity events to propagate. The staircase patterns may reduce the ion heat transport and a manipulation of these patterns may be used to optimize heat transport in tokamaks. An additional objective of the research in support of the Edge Physics Simulation initiative has been to improve the understanding of scrape-off layer thermal transport. In planning experiments and designing future tokamaks, it is important to understand the physical effects that contribute to divertor heat-load fluxes. The research accomplished will contribute to developing new models for the scrape-off layer region. The XGC0 code was used to compute the heat fluxes and the heat-load width in the outer divertor plates of C-Mod and DIII-D tokamaks. It was observed that the width of the XGC0 neoclassical heat-load was approximately inversely proportional to the total plasma current. Anomalous transport in the H-mode pedestal region of five Alcator C-Mod discharges, representing a collisionality scan, was analyzed. The understanding of anomalous transport in the pedestal region is important for the development of a comprehensive model for the H-mode pedestal slope. It was found that the electron thermal anomalous diffusivities at the pedestal top increase with the electron collisionality. This dependence can point to the DRIBM as the modes that drive the anomalous transport in the plasma edge of highly collisional discharges. The effects of plasma shaping on the H-mode pedestal structure was also investigated. The differences in the predicted H-mode pedestal width and height for the DIII-D discharges with different elongation and triangularities were discussed. For the discharges with higher elongation, it was found that the gradients of the plasma profiles in the H-mode pedestal reach semi-steady states. In these simulations, the pedestal slowly continued to evolve to higher pedestal pressures and bootstrap currents until the peeling ballooning stability conditions were satisfied. The discharges with lower elongation do not reach the semi-steady state, and ELM crashes were triggered at earlier times. The plasma elongation was found to have a stronger stabilizing effect than the plasma triangularity. For the discharges with lower elongation and lower triangularity, the ELM frequency was large, and the H-mode pedestal evolves rapidly. It was found that the temperature of neutrals in the scrape-off-layer region can affect the dynamics of the H-mode pedestal buildup. However, the final pedestal profiles were nearly independent of the neutral temperature. The elongation and triangularity affected the pedestal widths of plasma density and electron temperature profiles differently. This study illustrated a new mechanism for controlling the pedestal bootstrap current and the pedestal stability.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2015

2nd Generation QUATARA Flight Computer Project

Single core flight computer boards have been designed, developed, and tested (DD&T) to be flown in small satellites for the last few years. In this project, a prototype flight computer will be designed as a distributed multi-core system containing four microprocessors running code in parallel. This flight computer will be capable of performing multiple computationally intensive tasks such as processing digital and/or analog data, controlling actuator systems, managing cameras, operating robotic manipulators and transmitting/receiving from/to a ground station. In addition, this flight computer will be designed to be fault tolerant by creating both a robust physical hardware connection and by using a software voting scheme to determine the processor's performance. This voting scheme will leverage on the work done for the Space Launch System (SLS) flight software. The prototype flight computer will be constructed with Commercial Off-The-Shelf (COTS) components which are estimated to survive for two years in a low-Earth orbit.

Falker, Jay↗

DOE OSTI2014

Scaling Graph Community Detection on the Tilera Many-core Architecture

In an era when power constraints and data movement are proving to be significant barriers for the application of high-end computing, the Tilera many-core architecture offers a low-power platform exhibiting many important characteristics of future systems, including a large number of simple cores, a sophisticated network-on-chip, and fine-grained control over memory and caching policies. While this emerging architecture has been previously studied for structured compute-intensive kernels, benchmarking the platform for data-bound, irregular applications present significant challenges that have remained unexplored. Community detection is an advanced prototypical graph-theoretic operation with applications in numerous scientific domains including life sciences, cyber security, and power systems. In this work, we explore multiple design strategies toward developing a scalable tool for community detection on the Tilera platform. Using several memory layout and work scheduling techniques we demonstrate speedups of up to 46x on 36 cores of the Tilera TileGX36 platform over the best serial implementation, and also show results that have comparable quality and performance to mainstream x86 platforms. To the best of our knowledge this is the first work addressing graph algorithms on the Tilera platform. This study demonstrates that through careful design space exploration, low-power many-core platforms like Tilera can be effectively exploited for graph algorithms that that embody all the essential characteristics of an irregular application.

community detection↗

NASA NTRS1988

Implementing direct, spatially isolated problems on transputer networks

Parametric studies were performed on transputer networks of up to 40 processors to determine how to implement and maximize the performance of the solution of problems where no processor-to-processor data transfer is required for the problem solution (spatially isolated). Two types of problems are investigated a computationally intensive problem where the solution required the transmission of 160 bytes of data through the parallel network, and a communication intensive example that required the transmission of 3 Mbytes of data through the network. This data consists of solutions being sent back to the host processor and not intermediate results for another processor to work on. Studies were performed on both integer and floating-point transputers. The latter features an on-chip floating-point math unit and offers approximately an order of magnitude performance increase over the integer transputer on real valued computations. The results indicate that a minimum amount of work is required on each node per communication to achieve high network speedups (efficiencies). The floating-point processor requires approximately an order of magnitude more work per communication than the integer processor because of the floating-point unit's increased computing capacity.

Ellis, Graham K.↗

DOE OSTI2024

Parallel IO Libraries for Managing HEP Experimental Data

The computing and storage requirements of the energy and intensity frontiers will grow significantly during the Run 4 & 5 and the HL-LHC era. Similarly, in the intensity frontier, with larger trig ger readouts during supernovae explosions, the Deep Underground Neutrino Experiment (DUNE) will have unique computing challenges that could be addressed by the use of parallel and accelerated dataprocessing capabilities. Most of the requirements of the energy and intensity frontier experiments rely on increasing the role of high performance computing (HPC) in the HEP community. In this presentation, we will describe our ongoing efforts that are focused on using HPC resources for the next generation HEP experiments. The HEPCCE (High Energy Physics-Center for Computational Excellence) IOS (Input/Output and Storage) group has been developing approaches to map HEP data to the HDF5 , an IO library optimized for the HPC platforms to store the intermediate HEP data. The complex HEP data products are serialized using ROOT to allow for experiment independent general mapping approaches of the HEP data to the HDF5 format. The mapping approaches can be optimized for high performance parallel IO. Similarly, simpler data can be directly mapped into the HDF5, which can also be suitable for offloading into the GPUs directly. We will present our works on both complex and simple data model models.

Bashyal, Amit↗