Search NASA⌕ Search

SEARCH · Search NASA

Results for “data parallel”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Asi Nuclear Energy Sensors Data Portal Chatbot And Data Structuring Tool

The Idaho National Laboratory (INL) is advancing the development of an AI-powered chatbot and data structuring tool specifically designed to accelerate data mining processes for sensor-related information and seamlessly integrate the results into the ASI Sensors Data Portal (https://nes.energy.gov/). By doing so, the software aims to enhance the accessibility, usability, and organization of sensor data for nuclear energy applications. The software initial phase focuses on retrieving comprehensive datasets, prioritizing the past five years of publicly available information from the Office of Scientific and Technical Information (OSTI). These datasets will be meticulously processed to ensure compatibility, employing cleaning and preprocessing steps to eliminate irrelevant, incomplete, or corrupted information, thus establishing a robust foundation for subsequent AI use. The data will serve as the backbone for training an AI model and chatbot, which will act as an interactive tool enabling users to ask complex, context-specific questions and receive accurate, validated answers derived from constrained literature. In parallel, the project incorporates a data structuring process supported by AI to organize sensor information from multiple sources into a standardized format. This structured data will include detailed sensor specifications, such as measurement range, applications, accuracy, and operating conditions, generated and documented with AI. These specifications will be systematically integrated into the sensor portal. To maintain the highest levels of accuracy and relevance, all AI-generated outputs will be reviewed and validated by subject matter experts (SMEs), with additional fields or parameters added as needed. Future stages of the project aim to expand the dataset beyond OSTI to include other sources and potentially incorporate unclassified controlled information (UCI) with restricted access protocols to address security and confidentiality requirements.

Mapes, NormanJ. [Idaho National Laboratory (INL), ↗
DOE OSTI2016

In situ and in-transit analysis of cosmological simulations

Abstract Modern cosmological simulations have reached the trillion-element scale, rendering data storage and subsequent analysis formidable tasks. To address this circumstance, we present a new MPI-parallel approach for analysis of simulation data while the simulation runs, as an alternative to the traditional workflow consisting of periodically saving large data sets to disk for subsequent ‘offline’ analysis. We demonstrate this approach in the compressible gasdynamics/ N -body code Nyx, a hybrid $\mbox{MPI}+\mbox{OpenMP}$ MPI + OpenMP code based on the BoxLib framework, used for large-scale cosmological simulations. We have enabled on-the-fly workflows in two different ways: one is a straightforward approach consisting of all MPI processes periodically halting the main simulation and analyzing each component of data that they own (‘ in situ ’). The other consists of partitioning processes into disjoint MPI groups, with one performing the simulation and periodically sending data to the other ‘sidecar’ group, which post-processes it while the simulation continues (‘in-transit’). The two groups execute their tasks asynchronously, stopping only to synchronize when a new set of simulation data needs to be analyzed. For both the in situ and in-transit approaches, we experiment with two different analysis suites with distinct performance behavior: one which finds dark matter halos in the simulation using merge trees to calculate the mass contained within iso-density contours, and another which calculates probability distribution functions and power spectra of various fields in the simulation. Both are common analysis tasks for cosmology, and both result in summary statistics significantly smaller than the original data set. We study the behavior of each type of analysis in each workflow in order to determine the optimal configuration for the different data analysis algorithms.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

An in situ Comparison of Electron Acceleration at Collisionless Shocks under Differing Upstream Magnetic Field Orientations

A leading explanation for the origin of Galactic cosmic rays is acceleration at high-Mach number shock waves in the collisionless plasma surrounding young supernova remnants. Evidence for this is provided by multi-wavelength non-thermal emission thought to be associated with ultrarelativistic electrons at these shocks. However, the dependence of the electron acceleration process on the orientation of the upstream magnetic field with respect to the local normal to the shock front (quasi-parallel/quasi-perpendicular) is debated. Cassini spacecraft observations at Saturn’s bow shock have revealed examples of electron acceleration under quasi-perpendicular conditions, and the first in situ evidence of electron acceleration at a quasi-parallel shock. Here we use Cassini data to make the first comparison between energy spectra of locally accelerated electrons under these differing upstream magnetic field regimes. We present data taken during a quasi-perpendicular shock crossing on 2008 March 8 and during a quasi-parallel shock crossing on 2007 February 3, highlighting that both were associated with electron acceleration to at least MeV energies. The magnetic signature of the quasi-perpendicular crossing has a relatively sharp upstream–downstream transition, and energetic electrons were detected close to the transition and immediately downstream. The magnetic transition at the quasi-parallel crossing is less clear, energetic electrons were encountered upstream and downstream, and the electron energy spectrum is harder above ∼100 keV. We discuss whether the acceleration is consistent with diffusive shock acceleration theory in each case, and suggest that the quasi-parallel spectral break is due to an energy-dependent interaction between the electrons and short, large-amplitude magnetic structures.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2016

SPHERICAL HARMONIC ANALYSES OF INTENSITY MAPPING POWER SPECTRA

Intensity mapping is a promising technique for surveying the large-scale structure of our universe from z = 0 to z ∼ 150, using the brightness temperature field of spectral lines to directly observe previously unexplored portions of our cosmic timeline. Examples of targeted lines include the 21 cm hyperfine transition of neutral hydrogen, rotational lines of carbon monoxide, and fine-structure lines of singly ionized carbon. Recent efforts have focused on detections of the power spectrum of spatial fluctuations, but have been hindered by systematics such as foreground contamination. This has motivated the decomposition of data into Fourier modes perpendicular and parallel to the line of sight, which has been shown to be a particularly powerful way to diagnose systematics. However, such a method is well-defined only in the limit of a narrow-field, flat-sky approximation. This limits the sensitivity of intensity mapping experiments, as it means that wide surveys must be separately analyzed as a patchwork of smaller fields. In this paper, we develop a framework for analyzing intensity mapping data in a spherical Fourier–Bessel basis, which incorporates curved sky effects without difficulty. We use our framework to generalize a number of techniques in intensity mapping data analysis from the flat sky to the curved sky. These include visibility-based estimators for the power spectrum, treatments of interloper lines, and the “foreground wedge” signature of spectrally smooth foregrounds.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Stochastic AC optimal power flow: A data-driven approach

There is an emerging need for efficient solutions to stochastic AC Optimal Power Flow (AC-OPF) to ensure optimal and reliable grid operations in the presence of increasing demand and generation uncertainty. Herein this paper presents a highly scalable data-driven algorithm for stochastic AC-OPF that has extremely low sample requirement. The novelty behind the algorithm’s performance involves an iterative scenario design approach that merges information regarding constraint violations in the system with data-driven sparse regression. Compared to conventional methods with random scenario sampling, our approach is able to provide feasible operating points for realistic systems with much lower sample requirements. Furthermore, multiple sub-tasks in our approach can be easily paralleled and based on historical data to enhance its performance and application. We demonstrate the computational improvements of our approach through simulations on different test cases in the IEEE PES PGLib-OPF benchmark library.

42 ENGINEERING↗
DOE OSTI2016

Advances in targeted proteomics and applications to biomedical research

Targeted proteomics technique has emerged as a powerful protein quantification tool in systems biology, biomedical research, and increasing for clinical applications. The most widely used targeted proteomics approach, selected reaction monitoring (SRM), also known as multiple reaction monitoring (MRM), can be used for quantification of cellular signaling networks and preclinical verification of candidate protein biomarkers. As an extension to our previous review on advances in SRM sensitivity (Shi et al., Proteomics , 12, 1074–1092, 2012) herein we review recent advances in the method and technology for further enhancing SRM sensitivity (from 2012 to present), and highlighting its broad biomedical applications in human bodily fluids, tissue and cell lines. Furthermore, we also review two recently introduced targeted proteomics approaches, parallel reaction monitoring (PRM) and data‐independent acquisition (DIA) with targeted data extraction on fast scanning high‐resolution accurate‐mass (HR/AM) instruments. Such HR/AM targeted quantification with monitoring all target product ions addresses SRM limitations effectively in specificity and multiplexing; whereas when compared to SRM, PRM and DIA are still in the infancy with a limited number of applications. Thus, for HR/AM targeted quantification we focus our discussion on method development, data processing and analysis, and its advantages and limitations in targeted proteomics. Finally, general perspectives on the potential of achieving both high sensitivity and high sample throughput for large‐scale quantification of hundreds of target proteins are discussed.

Shi, Tujin↗
DOE OSTI2017

The HEP.TrkX Project: deep neural networks for HL-LHC online and offline tracking

Particle track reconstruction in dense environments such as the detectors of the High Luminosity Large Hadron Collider (HL-LHC) is a challenging pattern recognition problem. Traditional tracking algorithms such as the combinatorial Kalman Filter have been used with great success in LHC experiments for years. However, these state-of-the-art techniques are inherently sequential and scale poorly with the expected increases in detector occupancy in the HL-LHC conditions. The HEP.TrkX project is a pilot project with the aim to identify and develop cross-experiment solutions based on machine learning algorithms for track reconstruction. Machine learning algorithms bring a lot of potential to this problem thanks to their capability to model complex non-linear data dependencies, to learn effective representations of high-dimensional data through training, and to parallelize easily on high-throughput architectures such as GPUs. This contribution will describe our initial explorations into this relatively unexplored idea space. Furthermore, we will discuss the use of recurrent (LSTM) and convolutional neural networks to find and fit tracks in toy detector data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2016

Ultra-small-angle neutron scattering with azimuthal asymmetry

Small-angle neutron scattering (SANS) measurements from thin sections of rock samples such as shales demand as great a scattering vector range as possible because the pores cover a wide range of sizes. The limitation of the scattering vector range for pinhole SANS requires slit-smeared ultra-SANS (USANS) measurements that need to be converted to pinhole geometry. The desmearing algorithm is only successful for azimuthally symmetric data. Scattering from samples cut parallel to the plane of bedding is symmetric, exhibiting circular contours on a two-dimensional detector. Samples cut perpendicular to the bedding show elliptically dependent contours with the long axis corresponding to the normal to the bedding plane. A method is given for converting such asymmetric data collected on a double-crystal diffractometer for concatenation with the usual pinhole-geometry SANS data. Furthermore, the aspect ratio from the SANS data is used to modify the slit-smeared USANS data to produce quasi-symmetric contours. Rotation of the sample about the incident beam may result in symmetric data but cannot extract the same information as obtained from pinhole geometry.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Building a FAIR data ecosystem for incorporating single-cell transcriptomics data into agricultural genome to phenome research

Introduction The agriculture genomics community has numerous data submission standards available, but the standards for describing and storing single-cell (SC, e.g., scRNA- seq) data are comparatively underdeveloped. Methods To bridge this gap, we leveraged recent advancements in human genomics infrastructure, such as the integration of the Human Cell Atlas Data Portal with Terra, a secure, scalable, open-source platform for biomedical researchers to access data, run analysis tools, and collaborate. In parallel, the Single Cell Expression Atlas at EMBL-EBI offers a comprehensive data ingestion portal for high-throughput sequencing datasets, including plants, protists, and animals (including humans). Developing data tools connecting these resources would offer significant advantages to the agricultural genomics community. The FAANG data portal at EMBL-EBI emphasizes delivering rich metadata and highly accurate and reliable annotation of farmed animals but is not computationally linked to either of these resources. Results Herein, we describe a pilot-scale project that determines whether the current FAANG metadata standards for livestock can be used to ingest scRNA-seq datasets into Terra in a manner consistent with HCA Data Portal standards. Importantly, rich scRNA-seq metadata can now be brokered through the FAANG data portal using a semi-automated process, thereby avoiding the need for substantial expert curation. We have further extended the functionality of this tool so that validated and ingested SC files within the HCA Data Portal are transferred to Terra for further analysis. In addition, we verified data ingestion into Terra, hosted on Azure, and demonstrated the use of a workflow to analyze the first ingested porcine scRNA-seq dataset. Additionally, we have also developed prototype tools to visualize the output of scRNA-seq analyses on genome browsers to compare gene expression patterns across tissues and cell populations. This JBrowse tool now features distinct tracks, showcasing PBMC scRNA-seq alongside two bulk RNA-seq experiments. Discussion We intend to further build upon these existing tools to construct a scientist-friendly data resource and analytical ecosystem based on Findable, Accessible, Interoperable, and Reusable (FAIR) SC principles to facilitate SC-level genomic analysis through data ingestion, storage, retrieval, re-use, visualization, and comparative annotation across agricultural species.

Genetics & Heredity↗
DOE OSTI2021

Machine Learning Driven Contouring of High-Frequency Four-Dimensional Cardiac Ultrasound Data

Automatic boundary detection of 4D ultrasound (4DUS) cardiac data is a promising yet challenging application at the intersection of machine learning and medicine. Using recently developed murine 4DUS cardiac imaging data, we demonstrate here a set of three machine learning models that predict left ventricular wall kinematics along both the endo- and epi-cardial boundaries. Each model is fundamentally built on three key features: (1) the projection of raw US data to a lower dimensional subspace, (2) a smoothing spline basis across time, and (3) a strategic parameterization of the left ventricular boundaries. Model 1 is constructed such that boundary predictions are based on individual short-axis images, regardless of their relative position in the ventricle. Model 2 simultaneously incorporates parallel short-axis image data into their predictions. Model 3 builds on the multi-slice approach of model 2, but assists predictions with a single ground-truth position at end-diastole. To assess the performance of each model, Monte Carlo cross validation was used to assess the performance of each model on unseen data. For predicting the radial distance of the endocardium, models 1, 2, and 3 yielded average R2 values of 0.41, 0.49, and 0.71, respectively. Monte Carlo simulations of the endocardial wall showed significantly closer predictions when using model 2 versus model 1 at a rate of 48.67%, and using model 3 versus model 2 at a rate of 83.50%. These finding suggest that a machine learning approach where multi-slice data are simultaneously used as input and predictions are aided by a single user input yields the most robust performance. Subsequently, we explore the how metrics of cardiac kinematics compare between ground-truth contours and predicted boundaries. We observed negligible deviations from ground-truth when using predicted boundaries alone, except in the case of early diastolic strain rate, providing confidence for the use of such machine learning models for rapid and reliable assessments of murine cardiac function. To our knowledge, this is the first application of machine learning to murine left ventricular 4DUS data. Future work will be needed to strengthen both model performance and applicability to different cardiac disease models.

4D ultrasound↗
DOE OSTI2017

Collaborative simulations and experiments for a novel yield model of coal devolatilization in oxy-coal combustion conditions

Oxy-coal combustion is an emerging low-cost “clean coal” technology for emissions reduction and Carbon Capture and Sequestration (CCS). The use of Computational Fluid Dynamics (CFD) tools is crucial for the development of cost-effective oxy-fuel technologies and the minimization of environmental concerns at industrial scale. The coupling of detailed chemistry models and CFD simulations is still challenging, especially for large-scale plants, because of the high computational efforts required. The development of scale-bridging models is therefore necessary, to find a good compromise between computational efforts and the physical-chemical modeling precision. This paper presents a procedure for scale-bridging modeling of coal devolatilization, in the presence of experimental error, that puts emphasis on the thermodynamic aspect of devolatilization, namely the final volatile yield of coal, rather than kinetics. The procedure consists of an engineering approach based on dataset consistency and Bayesian methodology including Gaussian-Process Regression (GPR). Experimental data from devolatilization tests carried out in an oxy-coal entrained flow reactor were considered and CFD simulations of the reactor were performed. Jointly evaluating experiments and simulations, a novel yield model was validated against the data via consistency analysis. In parallel, a Gaussian-Process Regression was performed, to improve the understanding of the uncertainty associated to the devolatilization, based on the experimental measurements. Potential model forms that could predict yield during devolatilization were obtained. The set of model forms obtained via GPR includes the yield model that was proven to be consistent with the data. Finally, the overall procedure has resulted in a novel yield model for coal devolatilization and in a valuable evaluation of uncertainty in the data, in the model form, and in the model parameters.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2022

Proximity Portability and in Transit , M-to-N Data Partitioning and Movement in SENSEI [Book Chapter]

In high-performance parallel in situ processing, the term in transit processing refers to those configurations where data must move from a producer to a consumer that runs on separate resources. In the context of parallel and distributed computing on an HPC platform one of the central challenges is to determine a mapping of data from producer ranks to consumer ranks. This problem is complicated by the heterogeneity that arises in producer-consumer pairs, such as when producer and consumer codes have different levels of concurrency, different scaling characteristics, or different data models. The resulting mapping and movement of data from M producer to N consumer ranks can have a significant impact on aggregate application performance, particularly when the data consumer requires only a subset of the overall data for its task. This chapter focuses on the design considerations that underlie SENSEI’s implementation to this challenging problem. These design considerations extend the core SENSEI architecture and include ideas like the need to accommodate flexibility in the choice of different partitioning methods, the ability for a data consumer to request and receive only the subset of data needed for its particular operation, and the ability to leverage any of several different data transport tools. The idea of proximity portability, being able to use different data transport methods as part of an in transit workflow, is illustrated through the use of three different transport layers where switching from one transport tool to another is accomplished with only a configuration file change. Here, the chapter also includes a performance analysis summary showing the performance gains that are possible in terms of multiple metrics, such as memory footprint, time to solution, and amount of data moved, when using optimized partitioners in an in transit setting, gains that are made possible by the implementation shaped by specific design considerations.

Bethel, E. Wes↗
DOE OSTI2015

Morphologies of ∼190,000 galaxies at z = 0–10 revealed with HST legacy data. I. size evolution

We present the redshift evolution of the galaxy effective radius r{sub e} obtained from the Hubble Space Telescope (HST) samples of ∼190,000 galaxies at z = 0–10. Our HST samples consist of 176,152 photo-z galaxies at z = 0–6 from the 3D-HST+CANDELS catalog and 10,454 Lyman break galaxies (LBGs) at z = 4–10 identified in the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS), HUDF 09/12, and HFF parallel fields, providing the largest data set to date for galaxy size evolution studies. We derive r{sub e} with the same technique over the wide redshift range of z = 0–10, evaluating the optical-to-UV morphological K correction and the selection bias of photo-z galaxies+LBGs as well as the cosmological surface-brightness dimming effect. We find that r{sub e} values at a given luminosity significantly decrease toward high z, regardless of statistics choices (e.g., r{sub e}∝(1+z){sup −1.10±0.06} for median). For star-forming galaxies, there is no evolution of the power-law slope of the size–luminosity relation and the median Sérsic index (n∼1.5). Moreover, the r{sub e} distribution is well represented by log-normal functions whose standard deviation σ{sub lnr{sub e}} does not show significant evolution within the range of σ{sub lnr{sub e}}∼0.45−0.75. We calculate the stellar-to-halo size ratio from our r{sub e} measurements and the dark-matter halo masses estimated from the abundance-matching study, and we obtain a nearly constant value of r{sub e}/r{sub vir}=1.0%−3.5% at z = 0–8. The combination of the r{sub e}-distribution shape+standard deviation, the constant r{sub e}/r{sub vir}, and n∼1.5 suggests a picture in which typical high-z star-forming galaxies have disk-like stellar components in a sense of dynamics and morphology over cosmic time of z∼0−6. If high-z star-forming galaxies are truly dominated by disks, the r{sub e}/r{sub vir} value and the disk-formation model indicate that the specific angular momentum of the disk normalized by the host halo is j{sub d}/m{sub d}≃0.5−1. These are statistical results for major stellar components of galaxies, and the detailed study of clumpy subcomponents is presented in the paper II.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

The Improved Thermal Conductivity of a Potting Material for High-Power Fast Warm-Up Cathodes

With higher power, frequency and longer life of microwave tubes, cathode heater assembly is being modified to enhance the warm-up performance and reliability, which are governed by ceramics thermal conductivity, electrical resistivity and shrinkage. In present work, a potting material with a high thermal conductivity of 22.14 W/(m K) and low linear shrinkage of 5.5% was prepared using two kinds of alumina powders with different size and morphology. Aqueous slurries were produced for centrifugal casting. The highest solid loading slurries with 70 vol.% were obtained. Green body with relative density above 70% was easily made from slurries using centrifugal casting. Relative sintered density reached 86.75% maximum without any sintering aids. Rheological behavior of slurries, potting microstructure, sintering shrinkage and thermal conductivity were analyzed. Continuous particle distribution, series and parallel models were used to guide the powders proportion and explain the thermal conductivity. The potting material thermal conductivity experimental data agreed with the series and parallel models when the weight parameter ξ was between 0.3 and 0.5.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Pass-efficient methods for compression of high-dimensional turbulent flow data

The future of high-performance computing, specifically on future Exascale computers, will presumably see memory capacity and bandwidth fail to keep pace with data generated, for instance, from massively parallel partial differential equation (PDE) systems. Current strategies proposed to address this bottleneck entail the omission of large fractions of data, as well as the incorporation of in situ compression algorithms to avoid overuse of memory. To ensure that post-processing operations are successful, this must be done in a way that a sufficiently accurate representation of the solution is stored. Moreover, in situations where the input/output system becomes a bottleneck in analysis, visualization, etc., or the execution of the PDE solver is expensive, the number of passes made over the data must be minimized. In the interest of addressing this problem, this work focuses on the utility of pass-efficient, parallelizable, low-rank, matrix decomposition methods in compressing high-dimensional simulation data from turbulent flows. Additionally, a particular emphasis is placed on using coarse representation of the data – compatible with the PDE discretization grid – to accelerate the construction of the low-rank factorization. This includes the presentation of a novel single-pass matrix decomposition algorithm for computing the so-called interpolative decomposition. The methods are described extensively and numerical experiments on two turbulent channel flow data are performed. In the first (unladen) channel flow case, compression factors exceeding 400 are achieved while maintaining accuracy with respect to first- and second-order flow statistics. In the particle-laden case, compression factors of 100 are achieved and the compressed data is used to recover particle velocities. These results show that these compression methods can enable efficient computation of various quantities of interest in both the carrier and disperse phases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

ATLAS (Automatic Tool for Local Assembly Structures) - A Comprehensive Infrastructure for Assembly, Annotation, and Genomic Binning of Metagenomic and Metaranscripomic Data

ATLAS (Automatic Tool for Local Assembly Structures) is a comprehensive multiomics data analysis pipeline that is massively parallel and scalable. ATLAS contains a modular analysis pipeline for assembly, annotation, quantification and genome binning of metagenomics and metatranscriptomics data and a framework for reference metaproteomic database construction. ATLAS transforms raw sequence data into functional and taxonomic data at the microbial population level and provides genome-centric resolution through genome binning. ATLAS provides robust taxonomy based on majority voting of protein coding open reading frames rolled-up at the contig level using modified lowest common ancestor (LCA) analysis. ATLAS provides robust taxonomy based on majority voting of protein coding open reading frames rolled-up at the contig level using modified lowest common ancestor (LCA) analysis. ATLAS is user-friendly, easy install through bioconda maintained as open-source on GitHub, and is implemented in Snakemake for modular customizable workflows.

genome binning↗
DOE OSTI2018

FDTD: Solving 1+1D delay PDE in parallel

We present a proof of concept for solving a 1+1D complex-valued, delay partial differential equation (PDE) that emerges in the study of waveguide quantum electrodynamics (QED) by adapting the finite-difference time-domain (FDTD) method. The delay term is spatially non-local, rendering conventional approaches such as the method of lines inapplicable. Here, we show that by properly designing the grid and by supplying the (partial) exact solution as the boundary condition, the delay PDE can be numerically solved. In addition, we demonstrate that while the delay imposes strong data dependency, multi-thread parallelization can nevertheless be applied to such a problem. Our code provides a numerically exact solution to the time-dependent multi-photon scattering problem in waveguide QED.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Managing variations among nodes in parallel system frameworks

Systems, apparatuses, and methods for managing variations among nodes in parallel system frameworks. Sensor and performance data associated with the nodes of a multi-node cluster may be monitored to detect variations among the nodes. A variability metric may be calculated for each node of the cluster based on the sensor and performance data associated with the node. The variability metrics may then be used by a mapper to efficiently map tasks of a parallel application to the nodes of the cluster. In one embodiment, the mapper may assign the critical tasks of the parallel application to the nodes with the lowest variability metrics. In another embodiment, the hardware of the nodes may be reconfigured so as to reduce the node-to-node variability.

97 MATHEMATICS AND COMPUTING↗