Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

EJFAT: Towards Intelligent Compute Destination Load Balancing

To handle increased data flow, Jefferson Lab (JLab) is partnering with ESnet for development of an AI/ML directed compute work Load Balancer (LB) of UDP streamed data. The LB is FPGA based featuring dynamically configurable, low latency and high throughput destination address switching. The LB provides integration of edge and core computing to support JLab experimental programs, the Electron-Ion Collider, as well as data centers of the future. In the ESnet/JLab FPGA Accelerated Transport (EJFAT) initiative, the function of the LB Data Plane (DP) is to redirect data streams to selectable (but unknown to sender) destination hosts based on current worload and within that host to destination ports as a function of sub- stream id. This effects hierarchical scaling, first across compute machines for processing over a series of events and second, across ports so different data source sub-streams may be assigned to different processors for further parallelization. The LB Control Plane (CP) programs the DP using compute farm telemetry to direct and balance workloads across a compute cluster as the operating conditions require. While Proportional/Integrative/Derivative (PID) controllers are often seen in similar applications, here we investigate the feasibility of a Reinforcement Learning (RL) based schedule manager running in the CP to provide dynamic updates to the DP scheduling policy.

Lawrence, David↗
DOE OSTI2025

Unraveling the Determinant Mechanisms in Flow-Mediated Crystal Growth and Phase Behaviors

To uncover the critical mechanisms responsible for mesoscopic level development during flow-mediated crystal growth, we develop a semi-two-way hydrodynamic coupled structural phase-field crystal formalism (HXPFC-s2). The new formalism, inspired by previous attempts at coupling hydrodynamic and phase-field crystal (PFC) equations, allows for studying mesoscopic flow-mediated crystallization at diffusive timescales pertinent to industrial applications. Unlike previous efforts, the devised coupling to the structural PFC (XPFC) equations allows generalization to more complex crystal structures through explicit parameterization of the direct correlation function (DCF). Utilizing the HXPFC-s2 formalism, we seek to uncover the determinant physical mechanisms in crystallization under simple shear flows by comparing temperature-driven crystallization to flow-mediated crystallization under varying flow-strengths. Parallels and deviations of under-cooling and flow-strength effects on crystal growth are drawn using the crystal cluster-size and system ordering time evolutions. In doing so, we identify scaling behaviors with a Peclet-like number, Pe∼, a critical Peclet-like number, Pe∼*, and flow-field-crystal plane-dependent interactions. Our findings may be relevant for controlling crystal growth and phase behaviors in flow applications.

Willis, L. Connor (ORCID:0009000961321848)↗
NASA NTRS1980

Basic cluster compression algorithm

Feature extraction and data compression of LANDSAT data is accomplished by BCCA program which reduces costs associated with transmitting, storing, distributing, and interpreting multispectral image data. Algorithm uses spatially local clustering to extract features from image data to describe spectral characteristics of data set. Approach requires only simple repetitive computations, and parallel processing can be used for very high data rates. Program is written in FORTRAN IV for batch execution and has been implemented on SEL 32/55.

Hilbert, E. E.↗
NASA NTRS1979

X-ray measurements of stresses and defects in EFG and large grained polycrystalline silicon ribbons

The Bond method was employed to measure the lattice parameter in an area of 0.4 mm in diameter of EFG Si-ribbons to an accuracy of + or - 0.00008 A. A Bond goniometer was built which included a goniostat with a special specimen holder to mount ribbons 1 m in length and 75 mm in width which could be rotated about two orthogonal axes, and a Leitz microscope for precision alignment of a particular area into the center of the goniostat and the small primary X-ray beam. The (321) planes were found to be parallel to the surface of the ribbons with an angular spread of about 15 deg. The poles of the (111) planes clustered about an angle of 25 deg away from the surface normal, again with a spread of 10 deg. The lattice parameter of a small piece of ribbon material was found to be a sub o = 5.43075 A. A maximum stress of 115 MPa was observed in a fractured ribbon which corresponded to the fracture stress of single crystals of Si.

Wagner, C. N. J.↗
NASA NTRS2000

Parallel performance investigations of an unstructured mesh Navier-Stokes solver

A Reynolds-averaged Navier-Stokes solver based on unstructured mesh techniques for analysis of high-lift configurations is described. The method makes use of an agglomeration multigrid solver for convergence acceleration. Implicit line-smoothing is employed to relieve the stiffness associated with highly stretched meshes. A GMRES technique is also implemented to speed convergence at the expense of additional memory usage. The solver is cache efficient and fully vectorizable, and is parallelized using a two-level hybrid MPI-OpenMP implementation suitable for shared and/or distributed memory architectures, as well as clusters of shared memory machines. Convergence and scalability results are illustrated for various high-lift cases.

Mavriplis, Dimitri J.↗
NASA NTRS2001

Hyperspectral Image Classification using a Self-Organizing Map

The use of hyperspectral data to determine the abundance of constituents in a certain portion of the Earth's surface relies on the capability of imaging spectrometers to provide a large amount of information at each pixel of a certain scene. Today, hyperspectral imaging sensors are capable of generating unprecedented volumes of radiometric data. The Airborne Visible/Infrared Imaging Spectrometer (AVIRIS), for example, routinely produces image cubes with 224 spectral bands. This undoubtedly opens a wide range of new possibilities, but the analysis of such a massive amount of information is not an easy task. In fact, most of the existing algorithms devoted to analyzing multispectral images are not applicable in the hyperspectral domain, because of the size and high dimensionality of the images. The application of neural networks to perform unsupervised classification of hyperspectral data has been tested by several authors and also by us in some previous work. We have also focused on analyzing the intrinsic capability of neural networks to parallelize the whole hyperspectral unmixing process. The results shown in this work indicate that neural network models are able to find clusters of closely related hyperspectral signatures, and thus can be used as a powerful tool to achieve the desired classification. The present work discusses the possibility of using a Self Organizing neural network to perform unsupervised classification of hyperspectral images. In sections 3 and 4, the topology of the proposed neural network and the training algorithm are respectively described. Section 5 provides the results we have obtained after applying the proposed methodology to real hyperspectral data, described in section 2. Different parameters in the learning stage have been modified in order to obtain a detailed description of their influence on the final results. Finally, in section 6 we provide the conclusions at which we have arrived.

Martinez, P.↗
NASA NTRS1997

Predicting Cost/Performance Trade-Offs for Whitney: A Commodity Computing Cluster

Recent advances in low-end processor and network technology have made it possible to build a "supercomputer" out of commodity components. We develop simple models of the NAS Parallel Benchmarks version 2 (NPB 2) to explore the cost/performance trade-offs involved in building a balanced parallel computer supporting a scientific workload. We develop closed form expressions detailing the number and size of messages sent by each benchmark. Coupling these with measured single processor performance, network latency, and network bandwidth, our models predict benchmark performance to within 30%. A comparison based on total system cost reveals that current commodity technology (200 MHz Pentium Pros with 100baseT Ethernet) is well balanced for the NPBs up to a total system cost of around $1,000,000.

Becker, Jeffrey C.↗
NASA NTRS1998

Predicting Cost/Performance Trade-offs For Whitney: A Commodity Computing Cluster

Recent advances in low-end processor and network technology have made it possible to build a "supercomputer" out of commodity components. We develop simple models of the NAS Parallel Benchmarks version 2 (NPB 2) to explore the cost/performance trade-offs involved in building a balanced parallel computer supporting a scientific workload. By measuring single processor benchmark performance, network latency, and network bandwidth, and using closed form expressions detailing the number and size of messages sent by each benchmark, our models predict benchmark performance to within 30%. A comparison based on total system cost reveals that current commodity technology (200 MHz Pentium Pros with 100baseT Ethernet) is well balanced for the NPBs up to a total system cost of around $ 1,000,000.

Becker, Jeffrey C.↗
NASA NTRS2006

Processing EOS MLS Level-2 Data

A computer program performs level-2 processing of thermal-microwave-radiance data from observations of the limb of the Earth by the Earth Observing System (EOS) Microwave Limb Sounder (MLS). The purpose of the processing is to estimate the composition and temperature of the atmosphere versus altitude from .8 to .90 km. "Level-2" as used here is a specialists f term signifying both vertical profiles of geophysical parameters along the measurement track of the instrument and processing performed by this or other software to generate such profiles. Designed to be flexible, the program is controlled via a configuration file that defines all aspects of processing, including contents of state and measurement vectors, configurations of forward models, measurement and calibration data to be read, and the manner of inverting the models to obtain the desired estimates. The program can operate in a parallel form in which one instance of the program acts a master, coordinating the work of multiple slave instances on a cluster of computers, each slave operating on a portion of the data. Optionally, the configuration file can be made to instruct the software to produce files of simulated radiances based on state vectors formed from sets of geophysical data-product files taken as input.

Snyder, W. Van↗
NASA NTRS2011

EOS MLS Level 2 Data Processing Software Version 3

This software accepts the EOS MLS calibrated measurements of microwave radiances products and operational meteorological data, and produces a set of estimates of atmospheric temperature and composition. This version has been designed to be as flexible as possible. The software is controlled by a Level 2 Configuration File that controls all aspects of the software: defining the contents of state and measurement vectors, defining the configurations of the various forward models available, reading appropriate a priori spectroscopic and calibration data, performing retrievals, post-processing results, computing diagnostics, and outputting results in appropriate files. In production mode, the software operates in a parallel form, with one instance of the program acting as a master, coordinating the work of multiple slave instances on a cluster of computers, each computing the results for individual chunks of data. In addition, to do conventional retrieval calculations and producing geophysical products, the Level 2 Configuration File can instruct the software to produce files of simulated radiances based on a state vector formed from a set of geophysical product files taken as input. Combining both the retrieval and simulation tasks in a single piece of software makes it far easier to ensure that identical forward model algorithms and parameters are used in both tasks. This also dramatically reduces the complexity of the code maintenance effort.

Livesey, Nathaniel J.↗
DOE OSTI2024

The Effective Fragment Molecular Orbital Method: Achieving High Scalability and Accuracy for Large Systems

The effective fragment molecular orbital (EFMO) method has been developed to predict the total energy of a very large molecular system accurately (with respect to the underlying quantum mechanical method) and efficiently by taking advantage of the locality of strong chemical interactions and employing a two-level hierarchical parallelism. The accuracy of the EFMO method is partly attributed to the accurate and robust intermolecular interaction prediction between distant fragments, in particular, the many-body polarization and dispersion effects, which require the generation of static and dynamic polarizability tensors by solving the coupled perturbed Hartree–Fock (CPHF) and time-dependent HF (TDHF) equations, respectively. Solving the CPHF and TDHF equations is the main EFMO computational bottleneck due to the inefficient (serial) and I/O-intensive implementation of the CPHF and TDHF solvers. In this work, the efficiency and scalability of the EFMO method are significantly improved with a new CPU memory-based implementation for solving the CPHF and TDHF equations that are parallelized by either message passing interface (MPI) or hybrid MPI/OpenMP. Here, the accuracy of the EFMO method is demonstrated for both covalently bonded systems and noncovalently bound molecular clusters by systematically examining the effects of basis sets and a key distance-related cutoff parameter, R cut . R cut determines whether a fragment pair (dimer) is treated by the chosen ab initio method or calculated using the effective fragment potential (EFP) method (separated dimers). Decreasing the value of Rcut increases the number of separated (EFP) dimers, thereby decreasing the computational effort. It is demonstrated that excellent accuracy (<1 kcal/mol error per fragment) can be achieved when using a sufficiently large basis set with diffuse functions coupled with a small R cut value. With the new parallel implementation, the total EFMO wall time is substantially reduced, especially with a high number of MPI ranks. Given a sufficient workload, nearly ideal strong scaling is achieved for the CPHF and TDHF parts of the calculation. For the first time, EFMO calculations with the inclusion of long-range polarization and dispersion interactions on a hydrated mesoporous silica nanoparticle with explicit water solvent molecules (more than 15k atoms) are achieved on a massively parallel supercomputer using nearly 1000 physical nodes. In addition, EFMO calculations on the carbinolamine formation step of an amine-catalyzed aldol reaction at the nanoscale with explicit solvent effects are presented.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2019

GMRT 610 MHz observations of galaxy clusters in the ACT equatorial sample

Abstract We present Giant Metrewave Radio Telescope (GMRT) 610 MHz observations of 14 Atacama Cosmology Telescope (ACT) clusters, including new data for nine. The sample includes 73 per cent of ACT equatorial clusters with M500 > 5 × 1014 M⊙. We detect diffuse emission in three of these (27$^{+20}_{-14}$ per cent): we detect a radio minihalo in ACT-CL J0022.2–0036 at z = 0.8, making it the highest redshift minihalo known; we detect potential radio relic emission in ACT-CL J0014.9–0057 (z = 0.533); and we confirm the presence of a radio halo in low-mass cluster ACT-CL J0256.5+0006, with flux density S610 = 6.3 ± 0.4 mJy. We also detect residual diffuse emission in ACT-CL J0045.9–0152 (z = 0.545), which we cannot conclusively classify. For systems lacking diffuse radio emission, we determine radio halo upper limits in two ways and find via survival analysis that these limits do not significantly affect radio power scaling relations. Several clusters with no diffuse emission detection are known or suspected mergers, based on archival X-ray and/or optical measures; given the limited sensitivity of our observations, deeper observations of these disturbed systems are required in order to rule out the presence of diffuse emission consistent with known scaling relations. In parallel with our diffuse emission results, we present catalogues of individual radio sources, including a few interesting extended sources. Our study represents the first step towards probing the occurrence of diffuse emission in high-redshift (z ≳ 0.5) clusters, and serves as a pilot for statistical studies of larger cluster samples with the new radio telescopes available in the pre-SKA era.

Knowles, Kenda↗
DOE OSTI2016

Parallel Evolutionary Optimization for Neuromorphic Network Training

One of the key impediments to the success of current neuromorphic computing architectures is the issue of how best to program them. Evolutionary optimization (EO) is one promising programming technique; in particular, its wide applicability makes it especially attractive for neuromorphic architectures, which can have many different characteristics. In this paper, we explore different facets of EO on a spiking neuromorphic computing model called DANNA. We focus on the performance of EO in the design of our DANNA simulator, and on how to structure EO on both multicore and massively parallel computing systems. We evaluate how our parallel methods impact the performance of EO on Titan, the U.S.'s largest open science supercomputer, and BOB, a Beowulf-style cluster of Raspberry Pi's. We also focus on how to improve the EO by evaluating commonality in higher performing neural networks, and present the result of a study that evaluates the EO performed by Titan.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Mcqueuer

Mcqueuer is a simple tool that allows anyone from researchers to experienced developers to create multi-node/multi-core jobs by simply creating a file with a list of commands. Users simply combine tasks, which would otherwise each be their own job on the cluster, into a single file that is given to Mcqueuer. Mcqueuer then does the heavy lifting required to process the tasks in parallel in a single multi-node job. In addition, Mcqueuer provides load-balancing, which frees the user from having to worry about complex memory and CPU considerations, and instead focus on the processing itself.

Source record↗
DOE OSTI2012

A Path to Operating System and Runtime Support for Extreme Scale Tools

In this project, we cast distributed resource access as operations on files in a global name space and developed a common, scalable solution for group operations on distributed processes and files. The resulting solution enables tool and middleware developers to quickly create new scalable software or easily improve the scalability of existing software. The cornerstone of the project was the design of a new programming idiom called group file operations that eliminates iterative behavior when a single process must apply the same set of file operations to a group of related files. To demonstrate our novel and scalable ideas for group file operations and global name space composition, we developed a group file system called TBON-FS that leverages a tree-based overlay network (TBON), specifically MRNet, for logarithmic communication and distributed data aggregation. We also developed proc++, a new synthetic file system co-designed for use in scalable group file operations. Over the course of the project, we evaluated the utility and performance of group file operations, global name space composition, TBON-FS, and proc++ in three case studies. The first study focused on the ease in using group file operations and TBON-FS to quickly develop several new scalable tools for distributed system administration and monitoring. The second study evaluated the integration of group file operation and TBON-FS within the Ganglia Distributed Monitoring System to improve its scalability for clusters. The final study involved the integration of group file operations, TBON-FS, and proc++ within TotalView, the widely-used parallel debugger. For this project, the work of the Oak Ridge National Laboratory (ORNL) team occurred primarily in two directions: bringing the MRNet tree-based overlay network (TBON) implementation to the Cray XT platform, and investigating techniques for predicting the performance of MRNet topologies on such systems. Rogue Wave Software (RWS), formerly TotalView Technologies Inc., worked with the University ofWisconsin (UW) team on the design and prototyping of a scalable version of the TotalView debugger. RWS assisted UW with their "proc++" design effort. RWS assisted UW with strategy for integrating proc++ into TotalView.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1989

Scheduling Tasks In Parallel Processing

Algorithms sought to minimize time and cost of computation. Report describes research on scheduling of computations tasks in system of multiple identical data processors operating in parallel. Computational intractability requires use of suboptimal heuristic algorithms. First algorithm called "list heuristic", variation of classical list scheduling. Second algorithm called "cluster heuristic" applied to tightly coupled tasks and consists of four phases. Third algorithm called "exchange heuristic", iterative-improvement algorithm beginning with initial feasible assignment of tasks to processors and periods of time. Fourth algorithm is iterative one for optimal assignment of tasks and based on concept called "simulated annealing" because of mathematical resemblance to aspects of physical annealing processes.

Price, Camille C.↗
DOE OSTI2016

Mcqueuer

Mcqueuer is a simple tool that allows anyone from researchers to experienced developers to create multi-node/multi-core jobs by simply creating a file with a list of commands. Users simply combine tasks, which would otherwise each be their own job on the cluster, into a single file that is given to Mcqueuer. Mcqueuer then does the heavy lifting required to process the tasks in parallel in a single multi-node job. In addition, Mcqueuer provides load-balancing, which frees the user from having to worry about complex memory and CPU considerations, and instead focus on the processing itself.

Source record↗
DOE OSTI2016

Parallel Evolutionary Optimization for Neuromorphic Network Training

One of the key impediments to the success of current neuromorphic computing architectures is the issue of how best to program them. Evolutionary optimization (EO) is one promising programming technique; in particular, its wide applicability makes it especially attractive for neuromorphic architectures, which can have many different characteristics. In this paper, we explore different facets of EO on a spiking neuromorphic computing model called DANNA. We focus on the performance of EO in the design of our DANNA simulator, and on how to structure EO on both multicore and massively parallel computing systems. We evaluate how our parallel methods impact the performance of EO on Titan, the U.S.'s largest open science supercomputer, and BOB, a Beowulf-style cluster of Raspberry Pi's. We also focus on how to improve the EO by evaluating commonality in higher performing neural networks, and present the result of a study that evaluates the EO performed by Titan.

Schuman, Catherine D↗