Search NASA⌕ Search

SEARCH · Search NASA

Results for “clustering algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Accurate and efficient open-source implementation of domain-based local pair natural orbital (DLPNO) coupled-cluster theory using a t1-transformed Hamiltonian

We present an efficient, open-source formulation for coupled-cluster theory through perturbative triples with domain-based local pair natural orbitals [DLPNO-CCSD(T)]. Similar to the implementation of the DLPNO-CCSD(T) method found in the ORCA package, the most expensive integral generation and contraction steps associated with the CCSD(T) method are linear-scaling. In this work, we show that the t1-transformed Hamiltonian allows for a less complex algorithm when evaluating the local CCSD(T) energy without compromising efficiency or accuracy. Our algorithm yields sub-kJ mol−1 deviations for relative energies when compared with canonical CCSD(T), with typical errors being on the order of 0.1 kcal mol−1, using our TightPNO parameters. We extensively tested and optimized our algorithm and parameters for non-covalent interactions, which have been the most difficult interaction to model for orbital (PNO)-based methods historically. To highlight the capabilities of our code, we tested it on large water clusters, as well as insulin (787 atoms).

Chemistry↗
DOE OSTI2023

Q-BEEP: Quantum Bayesian Error Mitigation Employing Poisson Modeling over the Hamming Spectrum

Quantum computing technology has grown rapidly in recent years, with new technologies being explored, error rates being reduced, and quantum processor’s qubit capacity growing. However, near-term quantum algorithms are still unable to be induced without compounding consequential levels of noise, leading to non-trivial erroneous results. Quantum Error Correction (in-situ error mitigation) and Quantum Error Mitigation (post-induction error mitigation) are promising fields of research within the quantum algorithm scene, aiming to alleviate quantum errors, increasing the overall fidelity and hence the overall quality of circuit induction. Earlier this year, a pioneering work, namely HAMMER, published in ASPLOS-22 demonstrated the existence of a latent structure regarding post-circuit induction errors when mapping to the Hamming spectrum. However, they intuitively assumed that errors occur in local clusters, and that at higher average Hamming distances this structure falls away. In this work, we show that such a correlation structure is not only local but extends certain non-local clustering patterns which can be precisely described by a Poisson distribution model taking the input circuit, the device run time status (i.e., calibration statistics) and qubit topology into consideration. Using this quantum error characterizing model, we developed an iterative algorithm over the generated Bayesian network state-graph for post-induction error mitigation. Thanks to more precise modeling of the error distribution latent structure and the new iterative method, our Q-Beep approach provides state of the art performance and can boost circuit execution fidelity by up to 234.6% on Bernstein-Vazirani circuits and on average 71.0% on QAOA solution quality, using 16 practical IBMQ quantum processors. For other benchmarks such as those in QASMBench, the fidelity improvement is up to 17.8%. Q-Beep is a light-weight post-processing technique that can be performed offline and remotely, making it a useful tool for quantum vendors to integrate and provide more reliable circuit induction results.

Stein, Samuel A.↗
NASA NTRS2010

Deep Galex Observations of the Coma Cluster: Source Catalog and Galaxy Counts

We present a source catalog from deep 26 ks GALEX observations of the Coma cluster in the far-UV (FUV; 1530 Angstroms) and near-UV (NUV; 2310 Angstroms) wavebands. The observed field is centered 0.9 deg. (1.6 Mpc) south-west of the Coma core, and has full optical photometric coverage by SDSS and spectroscopic coverage to r-21. The catalog consists of 9700 galaxies with GALEX and SDSS photometry, including 242 spectroscopically-confirmed Coma member galaxies that range from giant spirals and elliptical galaxies to dwarf irregular and early-type galaxies. The full multi-wavelength catalog (cluster plus background galaxies) is 80% complete to NUV=23 and FUV=23.5, and has a limiting depth at NUV=24.5 and FUV=25.0 which corresponds to a star formation rate of 10(exp -3) solar mass yr(sup -1) at the distance of Coma. The GALEX images presented here are very deep and include detections of many resolved cluster members superposed on a dense field of unresolved background galaxies. This required a two-fold approach to generating a source catalog: we used a Bayesian deblending algorithm to measure faint and compact sources (using SDSS coordinates as a position prior), and used the GALEX pipeline catalog for bright and/or extended objects. We performed simulations to assess the importance of systematic effects (e.g. object blends, source confusion, Eddington Bias) that influence source detection and photometry when using both methods. The Bayesian deblending method roughly doubles the number of source detections and provides reliable photometry to a few magnitudes deeper than the GALEX pipeline catalog. This method is also free from source confusion over the UV magnitude range studied here: conversely, we estimate that the GALEX pipeline catalogs are confusion limited at NUV approximately 23 and FUV approximately 24. We have measured the total UV galaxy counts using our catalog and report a 50% excess of counts across FUV=22-23.5 and NUV=21.5-23 relative to previous GALEX measurements, which is not attributed to cluster member galaxies. Our galaxy counts are a better match to deeper UV counts measured with HST.

Hammer, D.↗
DOE OSTI2020

FIRED (Fire Events Delineation): An Open, Flexible Algorithm and Database of US Fire Events Derived from the MODIS Burned Area Product (2001–2019)

Harnessing the fire data revolution, i.e., the abundance of information from satellites, government records, social media, and human health sources, now requires complex and challenging data integration approaches. Defining fire events is key to that effort. In order to understand the spatial and temporal characteristics of fire, or the classic fire regime concept, we need to critically define fire events from remote sensing data. Events, fundamentally a geographic concept with delineated spatial and temporal boundaries around a specific phenomenon that is homogenous in some property, are key to understanding fire regimes and more importantly how they are changing. Here, we describe Fire Events Delineation (FIRED), an event-delineation algorithm, that has been used to derive fire events (N = 51,871) from the MODIS MCD64 burned area product for the coterminous US (CONUS) from January 2001 to May 2019. The optimized spatial and temporal parameters to cluster burned area pixels into events were an 11-day window and a 5-pixel (2315 m) distance, when optimized against 13,741 wildfire perimeters in the CONUS from the Monitoring Trends in Burn Severity record. The linear relationship between the size of individual FIRED and Monitoring Trends in Burn Severity (MTBS) events for the CONUS was strong (R2 = 0.92 for all events). Importantly, this algorithm is open-source and flexible, allowing the end user to modify the spatio-temporal threshold or even the underlying algorithm approach as they see fit. We expect the optimized criteria to vary across regions, based on regional distributions of fire event size and rate of spread. We describe the derived metrics provided in a new national database and how they can be used to better understand US fire regimes. The open, flexible FIRED algorithm could be utilized to derive events in any satellite product. We hope that this open science effort will help catalyze a community-driven, data-integration effort (termed OneFire) to build a more complete picture of fire.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1995

Parallel processing for scientific computations

The scope of this project dealt with the investigation of the requirements to support distributed computing of scientific computations over a cluster of cooperative workstations. Various experiments on computations for the solution of simultaneous linear equations were performed in the early phase of the project to gain experience in the general nature and requirements of scientific applications. A specification of a distributed integrated computing environment, DICE, based on a distributed shared memory communication paradigm has been developed and evaluated. The distributed shared memory model facilitates porting existing parallel algorithms that have been designed for shared memory multiprocessor systems to the new environment. The potential of this new environment is to provide supercomputing capability through the utilization of the aggregate power of workstations cooperating in a cluster interconnected via a local area network. Workstations, generally, do not have the computing power to tackle complex scientific applications, making them primarily useful for visualization, data reduction, and filtering as far as complex scientific applications are concerned. There is a tremendous amount of computing power that is left unused in a network of workstations. Very often a workstation is simply sitting idle on a desk. A set of tools can be developed to take advantage of this potential computing power to create a platform suitable for large scientific computations. The integration of several workstations into a logical cluster of distributed, cooperative, computing stations presents an alternative to shared memory multiprocessor systems. In this project we designed and evaluated such a system.

Alkhatib, Hasan S.↗
DOE OSTI2022

Machine learning and shallow groundwater chemistry to identify geothermal prospects in the Great Basin, USA

This study discovers various geothermal prospects in the Great Basin, USA based on shallow groundwater chemical (geochemical) data. The geochemical data are expected to include hidden (latent) information that is a proxy for geothermal prospectivity. We processed the sparse geochemical data in the Great Basin at 14,341 locations including 18 attributes. Next, a non-negative matrix factorization with customized k-means clustering is applied to the geochemical data matrix that automatically finds three hidden geothermal signatures representing modestly, moderately, and highly confident geothermal prospects. The algorithm also evaluated the probability of occurrence of these types of resources through the studied region. There is a consistency between regional geothermal prospectivity as estimated by our ML methodology and the traditional play fairway analysis conducted over a portion of the study area. We also identify the dominant data attributes associated with each signature. Finally, our ML analyses allow us to reconstruct attributes from sparse into continuous over the study domain. The predicted continuous attributes can be used for future detailed geothermal explorations in the Great Basin.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Topological grain boundary segregation transitions

Engineering the structure of grain boundaries (GBs) by solute segregation is a promising strategy to tailor the properties of polycrystalline materials. Solute segregation triggering phase transitions at GBs has been suggested theoretically to offer different pathways to design interfaces, but an understanding of their intrinsic atomistic nature is missing. Here, we combined atomic resolution electron microscopy and atomistic simulations to discover that iron segregation to GBs in titanium stabilizes icosahedral units (“cages”) that form robust building blocks of distinct GB phases. Owing to their five-fold symmetry, the iron cages cluster and assemble into hierarchical GB phases characterized by a different number and arrangement of the constituent icosahedral units. Our advanced GB structure prediction algorithms and atomistic simulations validate the stability of these observed phases and the high excess of iron at the GB that is accommodated by the phase transitions.

36 MATERIALS SCIENCE↗
DOE OSTI2023

HipBone: A performance-portable graphics processing unit-accelerated C++ version of the NekBone benchmark

We present hipBone, an open-source performance-portable proxy application for the Nek5000 (and NekRS) computational fluid dynamics applications. HipBone is a fully GPU-accelerated C++ implementation of the original NekBone CPU proxy application with several novel algorithmic and implementation improvements which optimize its performance on modern fine-grain parallel GPU accelerators. Our optimizations include a conversion to store the degrees of freedom of the problem in assembled form in order to reduce the amount of data moved during the main iteration and a portable implementation of the main Poisson operator kernel. We demonstrate near-roofline performance of the operator kernel on three different modern GPU accelerators from two different vendors. We present a novel algorithm for splitting the application of the Poisson operator on GPUs which aggressively hides MPI communication required for both halo exchange and assembly. Our implementation of nearest-neighbor MPI communication then leverages several different routing algorithms and GPU-Direct RDMA capabilities, when available, which improves scalability of the benchmark. We demonstrate the performance of hipBone on three different clusters housed at Oak Ridge National Laboratory, namely, the Summit supercomputer and the Frontier early-access clusters, Spock and Crusher. Our tests demonstrate both portability across different clusters and very good scaling efficiency, especially on large problems.

Computer Science↗
DOE OSTI2025

AI-powered topic modeling: comparing LDA and BERTopic in analyzing opioid-related cardiovascular risks in women

Topic modeling is a crucial technique in natural language processing (NLP), enabling the extraction of latent themes from large text corpora. Traditional topic modeling, such as Latent Dirichlet Allocation (LDA), faces limitations in capturing the semantic relationships in the text document although it has been widely applied in text mining. BERTopic, created in 2022, leveraged advances in deep learning and can capture the contextual relationships between words. In this work, we integrated Artificial Intelligence (AI) modules to LDA and BERTopic and provided a comprehensive comparison on the analysis of prescription opioid-related cardiovascular risks in women. Opioid use can increase the risk of cardiovascular problems in women such as arrhythmia, hypotension etc. 1,837 abstracts were retrieved and downloaded from PubMed as of April 2024 using three Medical Subject Headings (MeSH) words: “opioid,” “cardiovascular,” and “women.” Machine Learning of Language Toolkit (MALLET) was employed for the implementation of LDA. BioBERT was used for document embedding in BERTopic. Eighteen was selected as the optimal topic number for MALLET and 23 for BERTopic. ChatGPT-4-Turbo was integrated to interpret and compare the results. The short descriptions created by ChatGPT for each topic from LDA and BERTopic were highly correlated, and the performance accuracies of LDA and BERTopic were similar as determined by expert manual reviews of the abstracts grouped by their predominant topics. The results of the t-SNE (t-distributed Stochastic Neighbor Embedding) plots showed that the clusters created from BERTopic were more compact and well-separated, representing improved coherence and distinctiveness between the topics. Our findings indicated that AI algorithms could augment both traditional and contemporary topic modeling techniques. In addition, BERTopic has the connection port for ChatGPT-4-Turbo or other large language models in its algorithm for automatic interpretation, while with LDA interpretation must be manually, and needs special procedures for data pre-processing and stop words exclusion. Therefore, while LDA remains valuable for large-scale text analysis with resource constraints, AI-assisted BERTopic offers significant advantages in providing the enhanced interpretability and the improved semantic coherence for extracting valuable insights from textual data.

Research & Experimental Medicine↗
NASA NTRS1994

Performance Evaluation of Three Distributed Computing Environments for Scientific Applications

We present performance results for three distributed computing environments using the three simulated CFD applications in the NAS Parallel Benchmark suite. These environments are the DCF cluster, the LACE cluster, and an Intel iPSC/860 machine. The DCF is a prototypic cluster of loosely coupled SGI R3000 machines connected by Ethernet. The LACE cluster is a tightly coupled cluster of 32 IBM RS6000/560 machines connected by Ethernet as well as by either FDDI or an IBM Allnode switch. Results of several parallel algorithms for the three simulated applications are presented and analyzed based on the interplay between the communication requirements of an algorithm and the characteristics of the communication network of a distributed system.

Fatoohi, Rod↗
NASA NTRS1993

Discrete range clustering using Monte Carlo methods

For automatic obstacle avoidance guidance during rotorcraft low altitude flight, a reliable model of the nearby environment is needed. Such a model may be constructed by applying surface fitting techniques to the dense range map obtained by active sensing using radars. However, for covertness, passive sensing techniques using electro-optic sensors are desirable. As opposed to the dense range map obtained via active sensing, passive sensing algorithms produce reliable range at sparse locations, and therefore, surface fitting techniques to fill the gaps in the range measurement are not directly applicable. Both for automatic guidance and as a display for aiding the pilot, these discrete ranges need to be grouped into sets which correspond to objects in the nearby environment. The focus of this paper is on using Monte Carlo methods for clustering range points into meaningful groups. One of the aims of the paper is to explore whether simulated annealing methods offer significant advantage over the basic Monte Carlo method for this class of problems. We compare three different approaches and present application results of these algorithms to a laboratory image sequence and a helicopter flight sequence.

Chatterji, G. B.↗
NASA NTRS2000

[Compton GRO Cycle 7]

The goals of this project were to use the improved location accuracy of combined BATSE/Ulysses-IPN gamma-ray burst (GRB) data to search for small-angle clustering within the burst population and to search for possible correlations between bursts and catalogs of known astrophysical objects. The research was completed in several phases due to the length of time needed to process the basic BATSE/Ulysses data and to develop the appropriate analysis algorithms. In the first phase, BATSE/Ulysses data were gathered from their respective sources and processed to yield, final BATSE burst catalog locations and final BATSE/Ulysses IPN timing annuli. The final data are published. In the next phase, traditional statistical analysis techniques were used to search a subset of the data for small-angle clustering and. correlations, and to produce upper limits. Next, the unique analysis techniques and software described in the proposal were developed and,applied to the final set of data (corresponding to the 4B catalog). The final search did not find any significant self-clustering or correlations with supernovae, AGN, or near by galaxy clusters. The investigation ruled-out earlier findings that indicated (with marginal significance) such results, and. provided stringent limits on the allowable fraction of bursts that are self-clustered and associated with selected known objects. An added benefit of this research came in 1998 when it was suggested that some gamma-ray bursts may be associated with supernova (SN) explosions. We responded rapidly and applied the tools of this investigation to study this possibility. Using combined BATSE/Ulysses data, we were able to completely rule out any association between bright bursts and known supernovae, thereby questioning the validity of any GRB/SN association. If a GRB/SN association exists, it must be only with a small fraction of weak bursts.

Kippen, R. Marc↗
DOE OSTI2021

The Atacama Cosmology Telescope: SZ-based masses and dust emission from IR-selected cluster candidates in the SHELA survey

Here we examine the stacked thermal Sunyaev–Zel’dovich (SZ) signals for a sample of galaxy group and cluster candidates from the 24 deg 2 infrared Spitzer-HETDEX Exploratory Large Area (SHELA) survey. We identify the objects in combination with optical data using the redMaPPer algorithm, and divide them into three richness bins (λ in 10–20, 20–30, and 30–76 with average photometric redshifts of 0.80, 0.73, and 0.70, respectively). All richness bins show evidence for dust emission, which we fit using stacked profiles from Herschel Stripe 82 data. We fit for synchrotron emission using stacked profiles created by binning source fluxes from NRAO VLA Sky Survey data. We can confidently detect the SZ decrement only in the highest richness bin, finding M SZ,500 = $8.7^{+1.7}_{-1.3} \times 10^{13}\, \mathrm{ M}_\odot$. Neglecting the correction for dust and synchrotron depresses the inferred mass by 26 percent, indicating a partial fill-in of the SZ decrement from dust and synchrotron emission. We compare our corrected SZ masses to two redMaPPer mass–richness scaling relations and find that the SZ mass is lower than predicted by the richness. For the lower richness bins, mass bias factors as low as 1 – b = 0.6 are not enough to bring the mass limits into agreement. We discuss possible explanations for this discrepancy. The SHELA richnesses may differ from previous richness measurements due to the inclusion of infrared data in redMaPPer. To connect the SZ signal to the mass, we use a universal gas pressure profile that is calibrated to massive clusters at low redshift. It may not be applicable to our lower mass, higher redshift sample.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Flexible silicon photonic architecture for accelerating distributed deep learning

The increasing size and complexity of deep learning (DL) models have led to the wide adoption of distributed training methods in datacenters (DCs) and high-performance computing (HPC) systems. However, communication among distributed computing units (CUs) has emerged as a major bottleneck in the training process. In this study, we propose Flex-SiPAC, a flexible silicon photonic accelerated compute cluster designed to accelerate multi-tenant distributed DL training workloads. Flex-SiPAC takes a co-design approach that combines a silicon photonic hardware platform with a tailored collective algorithm, optimized to leverage the unique physical properties of the architecture. The hardware platform integrates a novel wavelength-reconfigurable transceiver design and a micro-resonator-based wavelength-reconfigurable switch, enabling the system to achieve flexible bandwidth steering in the wavelength domain. The collective algorithm is designed to support reconfigurable topologies, enabling efficient all-reduce communications that are commonly used in DL training. The feasibility of the Flex-SiPAC architecture is demonstrated through two testbed experiments. First, an optical testbed experiment demonstrates the flexible routing of wavelengths by shuffling an array of input wavelengths using a custom-designed spatial-wavelength selective switch. Second, a four-GPU testbed running two DL workloads shows a 23% improvement in job completion time compared to a similarly sized leaf-spine topology. We further evaluate Flex-SiPAC using large-scale simulations, which show that Flex-SiPAC is able to reduce the communication time by 26% to 29% compared to state-of-the-art compute clusters under representative collective operations.

Wu, Zhenguo (ORCID:0000000322847985)↗
DOE OSTI2025

DESI peculiar velocity survey – Fundamental Plane

The Dark Energy Spectroscopic Instrument (DESI) peculiar velocity survey aims to measure the peculiar velocities of early- and late-type galaxies within the DESI footprint using both the Fundamental Plane and optical Tully–Fisher relations. Direct measurements of peculiar velocities can significantly improve constraints on the growth rate of structure, reducing uncertainty by a factor of approximately 2.5 at redshift 0.1 compared to the DESI Bright Galaxy Survey’s redshift space distortion measurements alone. We assess the quality of stellar velocity dispersion measurements from DESI spectroscopic data. These measurements, along with photometric data from the Legacy Survey, establish the Fundamental Plane relation and determine distances and peculiar velocities of early-type galaxies. During survey validation, we obtain spectra for 6698 unique early-type galaxies, up to a photometric redshift of 0.15. 64 per cent of observed galaxies (4267) have relative velocity dispersion errors below 10 per cent. This percentage increases to 75 per cent if we restrict our sample to galaxies with spectroscopic redshifts below 0.1. We use the measured central velocity dispersion, along with photometry from the DESI Legacy Imaging Surveys, to fit the Fundamental Plane parameters using a 3D Gaussian maximum likelihood algorithm that accounts for measurement uncertainties and selection cuts. In addition, we conduct zero-point calibration using the absolute distance measurements to the Coma cluster, leading to a value of the Hubble constant, H 0 = 76.05 ± 0.35 (statistical) ±0.49 (systematic Fundamental Plane) ±4.86 (statistical due to calibration) km s –1 Mpc –1 ⁠. This H 0 value is within 2σ of Planck cosmic microwave background results and within 1σ of other low-redshift distance indicator-based measurements.

cosmological parameters↗
DOE OSTI2025

ICAT: The Interactive Corpus Analysis Tool

The Interactive Corpus Analysis Tool (ICAT) is a Python library for creating dashboards to explore textual datasets and build simple binary classification models to help filter through them and focus on entries of interest. This tool uses a form of interactive machine learning (IML), a paradigm of “machine teaching” (Simard et al., 2017) that sits at the intersection of the fields of human computer interaction (HCI), visual analytics, and machine learning. The intent of ICAT is to allow subject matter experts (SME) with limited to no experience in machine learning to benefit from an iterative human-in-the-loop (HITL) approach to building their own model without needing to understand the details of the underlying algorithm. This interactivity is achieved by allowing the user to create features, label data points, and visually manipulate a representation of the features to manually cluster and investigate data, while a model is trained on the fly based on these actions. ICAT is built on top of the Panel (Holoviz, 2018) library, using a combination of Vega, a custom IPyWidget using D3, and ipyvuetify, and is intended to be used inside of a Jupyter environment.

Martindale, Nathan [Oak Ridge National Laboratory ↗
NASA NTRS2011

Defining Dynamic Route Structure

This poster describes a method for defining route structure from flight tracks. Dynamically generated route structures could be useful in guiding dynamic airspace configuration and helping controllers retain situational awareness under dynamically changing traffic conditions. Individual merge and diverge intersections between pairs of flights are identified, clustered, and grouped into nodes of a route structure network. Links are placed between nodes to represent major traffic flows. A parametric analysis determined the algorithm input parameters producing route structures of current day flight plans that are closest to todays airway structure. These parameters are then used to define and analyze the dynamic route structure over the course of a day for current day flight paths. Route structures are also compared between current day flight paths and more user preferred paths such as great circle and weather avoidance routing.

Zelinski, Shannon↗