Search NASA⌕ Search

SEARCH · Search NASA

Results for “clustering algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Performance Evaluation of Three Distributed Computing Environments for Scientific Applications

We present performance results for three distributed computing environments using the three simulated CFD applications in the NAS Parallel Benchmark suite. These environments are the DCF cluster, the LACE cluster, and an Intel iPSC/860 machine. The DCF is a prototypic cluster of loosely coupled SGI R3000 machines connected by Ethernet. The LACE cluster is a tightly coupled cluster of 32 IBM RS6000/560 machines connected by Ethernet as well as by either FDDI or an IBM Allnode switch. Results of several parallel algorithms for the three simulated applications are presented and analyzed based on the interplay between the communication requirements of an algorithm and the characteristics of the communication network of a distributed system.

Fatoohi, Rod↗
NASA NTRS1993

Discrete range clustering using Monte Carlo methods

For automatic obstacle avoidance guidance during rotorcraft low altitude flight, a reliable model of the nearby environment is needed. Such a model may be constructed by applying surface fitting techniques to the dense range map obtained by active sensing using radars. However, for covertness, passive sensing techniques using electro-optic sensors are desirable. As opposed to the dense range map obtained via active sensing, passive sensing algorithms produce reliable range at sparse locations, and therefore, surface fitting techniques to fill the gaps in the range measurement are not directly applicable. Both for automatic guidance and as a display for aiding the pilot, these discrete ranges need to be grouped into sets which correspond to objects in the nearby environment. The focus of this paper is on using Monte Carlo methods for clustering range points into meaningful groups. One of the aims of the paper is to explore whether simulated annealing methods offer significant advantage over the basic Monte Carlo method for this class of problems. We compare three different approaches and present application results of these algorithms to a laboratory image sequence and a helicopter flight sequence.

Chatterji, G. B.↗
NASA NTRS2000

[Compton GRO Cycle 7]

The goals of this project were to use the improved location accuracy of combined BATSE/Ulysses-IPN gamma-ray burst (GRB) data to search for small-angle clustering within the burst population and to search for possible correlations between bursts and catalogs of known astrophysical objects. The research was completed in several phases due to the length of time needed to process the basic BATSE/Ulysses data and to develop the appropriate analysis algorithms. In the first phase, BATSE/Ulysses data were gathered from their respective sources and processed to yield, final BATSE burst catalog locations and final BATSE/Ulysses IPN timing annuli. The final data are published. In the next phase, traditional statistical analysis techniques were used to search a subset of the data for small-angle clustering and. correlations, and to produce upper limits. Next, the unique analysis techniques and software described in the proposal were developed and,applied to the final set of data (corresponding to the 4B catalog). The final search did not find any significant self-clustering or correlations with supernovae, AGN, or near by galaxy clusters. The investigation ruled-out earlier findings that indicated (with marginal significance) such results, and. provided stringent limits on the allowable fraction of bursts that are self-clustered and associated with selected known objects. An added benefit of this research came in 1998 when it was suggested that some gamma-ray bursts may be associated with supernova (SN) explosions. We responded rapidly and applied the tools of this investigation to study this possibility. Using combined BATSE/Ulysses data, we were able to completely rule out any association between bright bursts and known supernovae, thereby questioning the validity of any GRB/SN association. If a GRB/SN association exists, it must be only with a small fraction of weak bursts.

Kippen, R. Marc↗
DOE OSTI2021

The Atacama Cosmology Telescope: SZ-based masses and dust emission from IR-selected cluster candidates in the SHELA survey

Here we examine the stacked thermal Sunyaev–Zel’dovich (SZ) signals for a sample of galaxy group and cluster candidates from the 24 deg 2 infrared Spitzer-HETDEX Exploratory Large Area (SHELA) survey. We identify the objects in combination with optical data using the redMaPPer algorithm, and divide them into three richness bins (λ in 10–20, 20–30, and 30–76 with average photometric redshifts of 0.80, 0.73, and 0.70, respectively). All richness bins show evidence for dust emission, which we fit using stacked profiles from Herschel Stripe 82 data. We fit for synchrotron emission using stacked profiles created by binning source fluxes from NRAO VLA Sky Survey data. We can confidently detect the SZ decrement only in the highest richness bin, finding M SZ,500 = $8.7^{+1.7}_{-1.3} \times 10^{13}\, \mathrm{ M}_\odot$. Neglecting the correction for dust and synchrotron depresses the inferred mass by 26 percent, indicating a partial fill-in of the SZ decrement from dust and synchrotron emission. We compare our corrected SZ masses to two redMaPPer mass–richness scaling relations and find that the SZ mass is lower than predicted by the richness. For the lower richness bins, mass bias factors as low as 1 – b = 0.6 are not enough to bring the mass limits into agreement. We discuss possible explanations for this discrepancy. The SHELA richnesses may differ from previous richness measurements due to the inclusion of infrared data in redMaPPer. To connect the SZ signal to the mass, we use a universal gas pressure profile that is calibrated to massive clusters at low redshift. It may not be applicable to our lower mass, higher redshift sample.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Flexible silicon photonic architecture for accelerating distributed deep learning

The increasing size and complexity of deep learning (DL) models have led to the wide adoption of distributed training methods in datacenters (DCs) and high-performance computing (HPC) systems. However, communication among distributed computing units (CUs) has emerged as a major bottleneck in the training process. In this study, we propose Flex-SiPAC, a flexible silicon photonic accelerated compute cluster designed to accelerate multi-tenant distributed DL training workloads. Flex-SiPAC takes a co-design approach that combines a silicon photonic hardware platform with a tailored collective algorithm, optimized to leverage the unique physical properties of the architecture. The hardware platform integrates a novel wavelength-reconfigurable transceiver design and a micro-resonator-based wavelength-reconfigurable switch, enabling the system to achieve flexible bandwidth steering in the wavelength domain. The collective algorithm is designed to support reconfigurable topologies, enabling efficient all-reduce communications that are commonly used in DL training. The feasibility of the Flex-SiPAC architecture is demonstrated through two testbed experiments. First, an optical testbed experiment demonstrates the flexible routing of wavelengths by shuffling an array of input wavelengths using a custom-designed spatial-wavelength selective switch. Second, a four-GPU testbed running two DL workloads shows a 23% improvement in job completion time compared to a similarly sized leaf-spine topology. We further evaluate Flex-SiPAC using large-scale simulations, which show that Flex-SiPAC is able to reduce the communication time by 26% to 29% compared to state-of-the-art compute clusters under representative collective operations.

Wu, Zhenguo (ORCID:0000000322847985)↗
DOE OSTI2025

DESI peculiar velocity survey – Fundamental Plane

The Dark Energy Spectroscopic Instrument (DESI) peculiar velocity survey aims to measure the peculiar velocities of early- and late-type galaxies within the DESI footprint using both the Fundamental Plane and optical Tully–Fisher relations. Direct measurements of peculiar velocities can significantly improve constraints on the growth rate of structure, reducing uncertainty by a factor of approximately 2.5 at redshift 0.1 compared to the DESI Bright Galaxy Survey’s redshift space distortion measurements alone. We assess the quality of stellar velocity dispersion measurements from DESI spectroscopic data. These measurements, along with photometric data from the Legacy Survey, establish the Fundamental Plane relation and determine distances and peculiar velocities of early-type galaxies. During survey validation, we obtain spectra for 6698 unique early-type galaxies, up to a photometric redshift of 0.15. 64 per cent of observed galaxies (4267) have relative velocity dispersion errors below 10 per cent. This percentage increases to 75 per cent if we restrict our sample to galaxies with spectroscopic redshifts below 0.1. We use the measured central velocity dispersion, along with photometry from the DESI Legacy Imaging Surveys, to fit the Fundamental Plane parameters using a 3D Gaussian maximum likelihood algorithm that accounts for measurement uncertainties and selection cuts. In addition, we conduct zero-point calibration using the absolute distance measurements to the Coma cluster, leading to a value of the Hubble constant, H 0 = 76.05 ± 0.35 (statistical) ±0.49 (systematic Fundamental Plane) ±4.86 (statistical due to calibration) km s –1 Mpc –1 ⁠. This H 0 value is within 2σ of Planck cosmic microwave background results and within 1σ of other low-redshift distance indicator-based measurements.

cosmological parameters↗
DOE OSTI2025

ICAT: The Interactive Corpus Analysis Tool

The Interactive Corpus Analysis Tool (ICAT) is a Python library for creating dashboards to explore textual datasets and build simple binary classification models to help filter through them and focus on entries of interest. This tool uses a form of interactive machine learning (IML), a paradigm of “machine teaching” (Simard et al., 2017) that sits at the intersection of the fields of human computer interaction (HCI), visual analytics, and machine learning. The intent of ICAT is to allow subject matter experts (SME) with limited to no experience in machine learning to benefit from an iterative human-in-the-loop (HITL) approach to building their own model without needing to understand the details of the underlying algorithm. This interactivity is achieved by allowing the user to create features, label data points, and visually manipulate a representation of the features to manually cluster and investigate data, while a model is trained on the fly based on these actions. ICAT is built on top of the Panel (Holoviz, 2018) library, using a combination of Vega, a custom IPyWidget using D3, and ipyvuetify, and is intended to be used inside of a Jupyter environment.

Martindale, Nathan [Oak Ridge National Laboratory ↗
NASA NTRS2011

Defining Dynamic Route Structure

This poster describes a method for defining route structure from flight tracks. Dynamically generated route structures could be useful in guiding dynamic airspace configuration and helping controllers retain situational awareness under dynamically changing traffic conditions. Individual merge and diverge intersections between pairs of flights are identified, clustered, and grouped into nodes of a route structure network. Links are placed between nodes to represent major traffic flows. A parametric analysis determined the algorithm input parameters producing route structures of current day flight plans that are closest to todays airway structure. These parameters are then used to define and analyze the dynamic route structure over the course of a day for current day flight paths. Route structures are also compared between current day flight paths and more user preferred paths such as great circle and weather avoidance routing.

Zelinski, Shannon↗
DOE OSTI2021

Productive Programming of Distributed Systems with the SHAD C++ Library

High-performance computing (HPC) is often perceived as a matter of making large-scale systems (e.g., clusters) run as fast as possible, regardless the required programming effort. However, the idea of "bringing HPC to the masses" has recently emerged. Inspired by this vision, we have designed SHAD, the Scalable High-performance Algorithms and Data-structures library. SHAD is open source software, written in C++, for C++ developers. Unlike other HPC libraries for distributed systems, which rely on SPMD models, SHAD adopts a shared-memory programming abstraction, to make C++ programmers feel at home. Underneath, SHAD manages tasking and data-movements, moving the computation where data resides and taking advantage of asynchrony to tolerate network latency. At the bottom of his stack, SHAD can interface with multiple runtime systems: this not only improves developer’s productivity, by hiding the complexity of such software and of the underlying hardware, but also greatly enhance code portability. Thanks to its abstraction layers, SHAD can indeed target different systems, ranging from laptops to HPC clusters, without any need for modifying the user-level code. We have prototyped and open-sourced the implementation of (a subset of) the C++ standard library (STL) targeting multi-node HPC clusters. Our work allows plain STL-based C++ code to scale on HPC systems, with no need for rewriting the code to exploit the complex hardware. SHAD is available under Apache v2 License at https://github.com/pnnl/SHAD. In this paper we overview the design of the SHAD library, depicting its main components: runtime systems abstractions for tasking; parallel and distributed data-structures; STL-compliant interfaces and algorithms.

Castellana, Vito G.↗
DOE OSTI2020

The Architecture of Metabolism Maximizes Biosynthetic Diversity in the Largest Class of Fungi

Abstract Ecological diversity in fungi is largely defined by metabolic traits, including the ability to produce secondary or “specialized” metabolites (SMs) that mediate interactions with other organisms. Fungal SM pathways are frequently encoded in biosynthetic gene clusters (BGCs), which facilitate the identification and characterization of metabolic pathways. Variation in BGC composition reflects the diversity of their SM products. Recent studies have documented surprising diversity of BGC repertoires among isolates of the same fungal species, yet little is known about how this population-level variation is inherited across macroevolutionary timescales. Here, we applied a novel linkage-based algorithm to reveal previously unexplored dimensions of diversity in BGC composition, distribution, and repertoire across 101 species of Dothideomycetes, which are considered the most phylogenetically diverse class of fungi and known to produce many SMs. We predicted both complementary and overlapping sets of clustered genes compared with existing methods and identified novel gene pairs that associate with known secondary metabolite genes. We found that variation among sets of BGCs in individual genomes is due to nonoverlapping BGC combinations and that several BGCs have biased ecological distributions, consistent with niche-specific selection. We observed that total BGC diversity scales linearly with increasing repertoire size, suggesting that secondary metabolites have little structural redundancy in individual fungi. We project that there is substantial unsampled BGC diversity across specific families of Dothideomycetes, which will provide a roadmap for future sampling efforts. Our approach and findings lend new insight into how BGC diversity is generated and maintained across an entire fungal taxonomic class.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Cooperative Load Scheduling for Multiple Aggregators Using Hierarchical ADMM

Demand response (DR) serves an important role in improving the efficiency and stability of power systems. In recent years, with advances in communication and smart device technologies, many aggregators have emerged to facilitate end customer participation in DR programs. These aggregators, equipped with customized optimal control algorithms, are capable of providing various grid services. Among them is load scheduling during DR events, namely following a load signal provided by the utility company while minimizing overall customer discomfort. However, as the number of aggregators keeps increasing, it becomes challenging for utility companies to conduct load scheduling for multiple aggregators and generate reference signals for each of them. This paper proposes an optimization framework using hierarchical alternating direction method of multipliers (H-ADMM) to optimally generate load following signals for multiple aggregators. Under this framework, utility and multiple aggregators work in a cooperative manner, aiming at minimizing an overall system cost from different levels of the power system hierarchy, while protecting user privacy. A case study has been conducted in a system with multiple aggregators, based on control of HVAC loads. Experimental results validate the effectiveness of the proposed algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

SHAPER: can you hear the shape of a jet?

The identification of interesting substructures within jets is an important tool for searching for new physics and probing the Standard Model at colliders. Many of these substructure tools have previously been shown to take the form of optimal transport problems, in particular the Energy Mover’s Distance (EMD). In this work, we show that the EMD is in fact the natural structure for comparing collider events, which accounts for its recent success in understanding event and jet substructure. We then present a Shape Hunting Algorithm using Parameterized Energy Reconstruction (SHAPER), which is a general framework for defining and computing shape-based observables. SHAPER generalizes N-jettiness from point clusters to any extended, parametrizable shape. This is accomplished by efficiently minimizing the EMD between events and parameterized manifolds of energy flows representing idealized shapes, implemented using the dual-potential Sinkhorn approximation of the Wasserstein metric. We show how the geometric language of observables as manifolds can be used to define novel observables with built-in infrared-and-collinear safety. We demonstrate the efficacy of the SHAPER framework by performing empirical jet substructure studies using several examples of new shape-based observables.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Adaptive Hierarchical Cyber Attack Detection and Localization in Active Distribution Systems

Development of a cyber security strategy for the active distribution systems is challenging due to the inclusion of distributed renewable energy generations. Here this paper proposes an adaptive hierarchical cyber attack detection and localization framework for distributed active distribution systems via analyzing electrical waveforms. Cyber attack detection is based on a sequential deep learning model, via which even minor cyber attacks can be identified. The two-stage cyber attack localization algorithm first estimates the cyber attack sub-region, and then localize the specified cyber attack within the estimated subregion. We propose a modified spectral clustering-based network partitioning method for the hierarchical cyber attack ‘coarse’ localization. Next, to further narrow down the cyber attack location, a normalized impact score based on waveform statistical metrics is proposed to obtain a ‘fine’ cyber attack location by characterizing different waveform properties. Finally, compared with classical and state-of-art methods, a comprehensive quantitative evaluation with two case studies shows promising estimation results of the proposed framework.

42 ENGINEERING↗
DOE OSTI2024

Augmented Human Analysis (AHA)

Radio frequency (RF) signal monitoring generally emphasizes intentionally generated signals, such as WiFi, Bluetooth, or cellular transmissions. However, electronic devices also produce unintended radiated emissions (UREs), which could also be useful in RF spectrum analysis. In either case, deriving intelligence from RF signals is typically a human-intensive process requiring significant domain knowledge. In the Augmented Human Analysis (AHA) project, we investigate the utility of dimensionally aligned signal projection (DASP) and machine learning (ML) algorithms for accelerating RF analysis workflows. We find that while DASP algorithms can indeed highlight signal characteristics relevant for classification tasks, the choice of algorithmic hyperparameters greatly affects performance. To address this challenge, we evaluate the quality of DASP outputs using the silhouette score, which measures how well data points cluster; high silhouette scores indicate good clustering, and thus good hyperparameter values. This approach is critical for machine learning pipelines as the DASP parameters cannot be directly optimized during model training. By identifying good DASP parameters, and thus good DASP outputs, as a preprocessing step, we can decrease the amount of effort required for downstream ML model training. We demonstrate our workflow using a dataset of UREs from common household devices, showing that even without the aid of ML, proper selection of DASP parameters enables clustering by device type.

42 ENGINEERING↗
NASA NTRS1977

Multispectral image processor

Correlation clustering of 250,000 pixels are numerically classified in real time according to various image elements. Processor operates upon data supplied by Earth Resources Technology Satellite. Algorithmic signal manipulation is used to provide discrete control of individual image parameters.

Haskell, R. E.↗
DOE OSTI2025

Spectral Clustering-Based Partitioning of Large-Scale Power Electronics-Based Power Systems for Small-Signal Stability Analysis

The nodal admittance matrix (NAM)-based approach is well-suited for small-signal stability analysis of large-scale power electronics-based power systems (PEPSs), as it preserves the system structure through its admittance matrix. Previous studies have explored partitioning such systems into subareas and interconnections to reduce computational burden; however, they lacked a formal algorithmic procedure for determining feasible partitions. While several grid partitioning methods, such as those based on graph theory or machine learning, exist in the literature, they cannot be directly applied to NAM-based analysis due to differing objectives and constraints. Here, this paper addresses this gap by presenting a systematic, step-by-step procedure for applying a spectral partitioning algorithm that yields a division of the system into subareas suitable for NAM-based analysis. The computational complexity of the proposed method is also derived to demonstrate its efficiency and justify the practicality of the resulting subarea decomposition. The performance of the partitioning method is evaluated by applying the spectral clustering-derived subareas and interconnections to the NAM-based partitioning approach on a 140-bus system. Computational times for the full-system and partitioned NAM analyses are compared using MATLAB. Additionally, PSCAD simulations of the complete system and partitioned subareas are carried out to verify the effectiveness of the proposed method.

Nupur [Univ. of Tennessee, Knoxville, TN (United S↗
DOE OSTI2022

Using machine learning to identify extragalactic globular cluster candidates from ground-based photometric surveys of M87

Globular clusters (GCs) have been at the heart of many longstanding questions in many sub-fields of astronomy and, as such, systematic identification of GCs in external galaxies has immense impacts. In this study, we take advantage of M87’s well-studied GC system to implement supervised machine learning (ML) classification algorithms – specifically random forest and neural networks – to identify GCs from foreground stars and background galaxies, using ground-based photometry from the Canada–France–Hawaii Telescope (CFHT). We compare these two ML classification methods to studies of ‘human-selected’ GCs and find that the best-performing random forest model can reselect 61.2 per cent ± 8.0 per cent of GCs selected from HST data (ACSVCS) and the best-performing neural network model reselects 95.0 per cent ± 3.4 per cent. When compared to human-classified GCs and contaminants selected from CFHT data – independent of our training data – the best-performing random forest model can correctly classify 91.0 per cent ± 1.2 per cent and the best-performing neural network model can correctly classify 57.3 per cent ± 1.1 per cent. ML methods in astronomy have been receiving much interest as Vera C. Rubin Observatory prepares for first light. The observables in this study are selected to be directly comparable to early Rubin Observatory data and the prospects for running ML algorithms on the upcoming data set yields promising results.

79 ASTRONOMY AND ASTROPHYSICS↗