SEARCH · Search NASA

Results for “cluster methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Clustering Analysis of Commercial Vehicles Using Automatically Extracted Features from Time Series Data

Standard of practice approaches to time series cluster analysis involve careful feature engineering, often utilizing expert input to tune and select features by hand. In many cases, expert input may not be readily available, or there may not yet exist a community consensus on the ideal features for a given application. This paper compares the results of several cluster analysis methods, using both hand selected features and those extracted automatically, when applied to large geospatial time series telematics data from commercial trucking fleets. The impacts of feature selection, dimensionality reduction, and choice of clustering algorithm on the quality of clustering results are explored. Results from this analysis confirm prior results that domain agnostic features are competitive with the hand engineered features with respect to clustering quality metrics. These results also provide new insight into the most successful strategies for identifying structure in large unstructured vehicle telematics data, and suggest that time series clustering using automatic feature extraction can be an effective approach to extract structure from large scale geospatial time series data in cases when hand selected features are not available.

33 ADVANCED PROPULSION SYSTEMS↗

NASA NTRS1985

Applications of Some Artificial Intelligence Methods to Satellite Soundings

Hard clustering of temperature profiles and regression temperature retrievals were used to refine the method using the probabilities of membership of each pattern vector in each of the clusters derived with discriminant analysis. In hard clustering the maximum probability is taken and the corresponding cluster as the correct cluster are considered discarding the rest of the probabilities. In fuzzy partitioned clustering these probabilities are kept and the final regression retrieval is a weighted regression retrieval of several clusters. This method was used in the clustering of brightness temperatures where the purpose was to predict tropopause height. A further refinement is the division of temperature profiles into three major regions for classification purposes. The results are summarized in the tables total r.m.s. errors are displayed. An approach based on fuzzy logic which is intimately related to artificial intelligence methods is recommended.

Munteanu, M. J.↗

DOE OSTI2021

Elaborated thermochemical treatment of HF, CO, N 2 , and H 2 O: Insight into HEAT and its extensions

Here, empirical, highly accurate non-relativistic electronic total atomization energies (eTAEs) are established by combining experimental or computationally converged treatments of the nuclear motion and relativistic contributions with the total atomization energies of HF, CO, N 2 , and H 2 O obtained from the Active Thermochemical Tables. These eTAEs, which have estimated (2σ) uncertainties of less than 10 cm -1 (0.12 kJ mol -1 ), form the basis for an analysis of high-level ab initio quantum chemical calculations that aim at reproducing these eTAEs for the title molecules. The results are then employed to analyze the performance of the high-accuracy extrapolated ab initio thermochemistry, or High-Accuracy Extrapolated Ab Initio Thermochemistry (HEAT), family of theoretical methods. The method known as HEAT-345(Q), in particular, is found to benefit from fortuitous error cancellation between its treatment of the zero-point energy, extrapolation errors in the Hartree-Fock and coupled cluster contributions, neglect of post-(T) core-correlation, and the basis-set error involved in higher-level correlation corrections. In addition to shedding light on a longstanding curiosity of the HEAT protocol—where the cheapest HEAT-345(Q) performs comparably to the theoretically more complete HEAT-456QP procedure—this study lays the foundation for extended HEAT variants that offer substantial improvements in accuracy relative to the established approaches.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRS1986

The cluster-cluster correlation function

The clustering properties of the Abell and Zwicky cluster catalogs are studied using the two-point angular and spatial correlation functions. The catalogs are divided into eight subsamples to determine the dependence of the correlation function on distance, richness, and the method of cluster identification. It is found that the Corona Borealis supercluster contributes significant power to the spatial correlation function to the Abell cluster sample with distance class of four or less. The distance-limited catalog of 152 Abell clusters, which is not greatly affected by a single system, has a spatial correlation function consistent with the power law Xi(r) = 300r exp -1.8. In both the distance class four or less and distance-limited samples the signal in the spatial correlation function is a power law detectable out to 60/h Mpc. The amplitude of Xi(r) for clusters of richness class two is about three times that for richness class one clusters. The two-point spatial correlation function is sensitive to the use of estimated redshifts.

Postman, M.↗

DOE OSTI2022

Automated Identification of Characteristic Droplet Size Distributions in Stratocumulus Clouds Utilizing a Data Clustering Algorithm

Abstract Droplet-level interactions in clouds are often parameterized by a modified gamma fitted to a “global” droplet size distribution. Do “local” droplet size distributions of relevance to microphysical processes look like these average distributions? This paper describes an algorithm to search and classify characteristic size distributions within a cloud. The approach combines hypothesis testing, specifically, the Kolmogorov–Smirnov (KS) test, and a widely used class of machine learning algorithms for identifying clusters of samples with similar properties: density-based spatial clustering of applications with noise (DBSCAN) is used as the specific example for illustration. The two-sample KS test does not presume any specific distribution, is parameter free, and avoids biases from binning. Importantly, the number of clusters is not an input parameter of the DBSCAN-type algorithms but is independently determined in an unsupervised fashion. As implemented, it works on an abstract space from the KS test results, and hence spatial correlation is not required for a cluster. The method is explored using data obtained from the Holographic Detector for Clouds (HOLODEC) deployed during the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. The algorithm identifies evidence of the existence of clusters of nearly identical local size distributions. It is found that cloud segments have as few as one and as many as seven characteristic size distributions. To validate the algorithm’s robustness, it is tested on a synthetic dataset and successfully identifies the predefined distributions at plausible noise levels. The algorithm is general and is expected to be useful in other applications, such as remote sensing of cloud and rain properties. Significance Statement A typical cloud can have billions of drops spread over tens or hundreds of kilometers in space. Keeping track of the sizes, positions, and interactions of all of these droplets is impractical, and, as such, information about the relative abundance of large and small drops is typically quantified with a “size distribution.” Droplets in a cloud interact locally, however, so this work is motivated by the question of whether the cloud droplet size distribution is different in different parts of a cloud. A new method, based on hypothesis testing and machine learning, determines how many different size distributions are contained in a given cloud. This is important because the size distribution describes processes such as cloud droplet growth and light transmission through clouds.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1987

Cluster analysis of multiple planetary flow regimes

A modified cluster analysis method was developed to identify spatial patterns of planetary flow regimes, and to study transitions between them. This method was applied first to a simple deterministic model and second to Northern Hemisphere (NH) 500 mb data. The dynamical model is governed by the fully-nonlinear, equivalent-barotropic vorticity equation on the sphere. Clusters of point in the model's phase space are associated with either a few persistent or with many transient events. Two stationary clusters have patterns similar to unstable stationary model solutions, zonal, or blocked. Transient clusters of wave trains serve as way stations between the stationary ones. For the NH data, cluster analysis was performed in the subspace of the first seven empirical orthogonal functions (EOFs). Stationary clusters are found in the low-frequency band of more than 10 days, and transient clusters in the bandpass frequency window between 2.5 and 6 days. In the low-frequency band three pairs of clusters determine, respectively, EOFs 1, 2, and 3. They exhibit well-known regional features, such as blocking, the Pacific/North American (PNA) pattern and wave trains. Both model and low-pass data show strong bimodality. Clusters in the bandpass window show wave-train patterns in the two jet exit regions. They are related, as in the model, to transitions between stationary clusters.

Mo, Kingtse↗

DOE OSTI2025

Reward Driven Workflows for Unsupervised Explainable Analysis of Phases and Ferroic Variants From Atomically Resolved Imaging Data

Rapid progress in aberration corrected electron microscopy necessitates development of robust methods for the identification of phases, ferroic variants, and other pertinent aspects of materials structure from imaging data. While unsupervised methods for clustering and classification are widely used for these tasks, their performance can be sensitive to hyperparameter selection in the analysis workflow. In this study, the effects of descriptors and hyperparameters are explored on the capability of unsupervised ML methods to distill local structural information, exemplified by the discovery of polarization and lattice distortion in Sm − dopped BiFeO 3 (BFO) thin films. It is demonstrated that a reward-driven approach can be used to optimize these key hyperparameters across the full workflow, where rewards are designed to reflect domain wall continuity and straightness, ensuring that the analysis aligns with the material's physical behavior. This approach allows the discovery of local descriptors that are best aligned with the specific physical behavior, providing insight into the fundamental physics of materials. The reward driven workflow is further extended to disentangle structural factors of variation via an optimized variational autoencoder (VAE). Lastly, the importance of well-defined rewards is explored as a quantifiable measure of the success of the workflow.

Barakati, Kamyar [University of Tennessee, Knoxvil↗

DOE OSTI2024

Stochastic economic dispatch of wind power under uncertainty using clustering-based extreme scenarios

Operation of power systems with high penetrations of renewable energy sources requires tools for robust decision making under uncertainty. Stochastic economic dispatch and stochastic unit commitment are effective techniques for planning and operation under uncertainty, whose effectiveness depends on the cardinality and quality of the scenario set. Here, this article proposes a machine learning method using -means clustering for capturing relevant physical information from a large population of analog scenarios. Extreme scenario samples drawn from the clusters are used in a two-stage stochastic economic dispatch computation. The effectiveness of the proposed approach is assessed on a synthetic 200-bus system with a geographic footprint over Illinois, USA for four months from each season of WIND Toolkit data. The combination of -means clustering with importance sampling is shown to reduce the total operational cost by over 43% compared to sampling from populations based on heuristic clustering-based methods. Additionally, the variability in the mean cost is about 56% lower than the variability using Monte Carlo sampling. Moreover, the operational cost with the presented approach is shown to be close to the cost calculated based on a hindsight exact wind profile, signifying a highly accurate quantification of wind uncertainty by the presented -means clustering based sampling method.

17 WIND ENERGY↗

NASA NTRS1982

Methods of editing cloud and atmospheric layer affected pixels from satellite data

Practical methods of computer screening cloud-contaminated pixels from data of various satellite systems are proposed. Examples are given of the location of clouds and representative landscape features in HCMM spectral space of reflectance (VIS) vs emission (IR). Methods of screening out cloud affected HCMM are discussed. The character of subvisible absorbing-emitting atmospheric layers (subvisible cirrus or SCi) in HCMM data is considered and radiosonde soundings are examined in relation to the presence of SCi. The statistical characteristics of multispectral meteorological satellite data in clear and SCi affected areas are discussed. Examples in TIROS-N and NOAA-7 data from several states and Mexico are presented. The VIS-IR cluster screening method for removing clouds is applied to a 262, 144 pixel HCMM scene from south Texas and northeast Mexico. The SCi that remain after cluster screening are sited out by applying a statistically determined IR limit.

Nixon, P. R.↗

DOE OSTI2025

Enhancing the accuracy of XPS calculations: Exploring hybrid basis set schemes for CVS-EOMIP-CCSD calculations

Reliable computational methodologies and basis sets for modeling x-ray spectra are essential for extracting and interpreting electronic and structural information from experimental x-ray spectra. In particular, the trade-off between numerical accuracy and computational cost due to the size of the basis set is a major challenge, since molecular orbitals undergo extreme relaxation in the core-hole state. To gain clarity on the changes in electronic structure induced by the formation of a core-hole, the use of sufficiently flexible basis for expanding the orbitals, particularly for the core region, has been shown to be essential. This work focuses on the refinement of core-hole ionized state calculations using the equation-of-motion coupled cluster family of methods through an extensive analysis on the effectiveness of “hybrid” and mixed basis sets. In this investigation, we utilize the CVS-EOMIP-CCSD method in combination and construct hybrid basis sets piecewise from readily available Dunning’s correlation consistent basis sets in order to calculate x-ray ionization energies (IEs) for a set of small gas phase molecules. Our results provide insights into the impact of basis sets on the CVS-EOMIP-CCSD calculations of K-edge IEs of first-row p-block elements. Furthermore, these insights enable us to understand more about the basis set dependence of the core IEs computed and allow us to establish a protocol for deriving reliable and cost-effective theoretical estimates for computing IEs of small molecules containing such elements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRS1975

Investigation of correlation classification techniques

A two-step classification algorithm for processing multispectral scanner data was developed and tested. The first step is a single pass clustering algorithm that assigns each pixel, based on its spectral signature, to a particular cluster. The output of that step is a cluster tape in which a single integer is associated with each pixel. The cluster tape is used as the input to the second step, where ground truth information is used to classify each cluster using an iterative method of potentials. Once the clusters have been assigned to classes the cluster tape is read pixel-by-pixel and an output tape is produced in which each pixel is assigned to its proper class. In addition to the digital classification programs, a method of using correlation clustering to process multispectral scanner data in real time by means of an interactive color video display is also described.

Haskell, R. E.↗

DOE OSTI2025

Lieb-Robinson Bounds with Exponential-in-Volume Tails

Lieb-Robinson bounds demonstrate the emergence of locality in many-body quantum systems. Intuitively, Lieb-Robinson bounds state that, with local or exponentially decaying interactions, the correlation that can be built up between two sites separated by distance 𝑟 after a time 𝑡 decays as exp (𝑣⁢𝑡 −𝑟), where 𝑣 is the emergent Lieb-Robinson velocity. In many problems, it is important to also capture how much of an operator grows to act on 𝑟 𝑑 sites in 𝑑 spatial dimensions. Perturbation theory and cluster expansion methods suggest that, at short times, these volume-filling operators are suppressed as exp (−𝑟 𝑑 ). We confirm this intuition, showing that, for 𝑟 >𝑣⁢𝑡, the volume-filling operator is suppressed by exp (−(𝑟−𝑣⁢𝑡) 𝑑 /(𝑣⁢𝑡) 𝑑−1 ). This closes a conceptual and practical gap between the cluster expansion and the Lieb-Robinson bound. We then present two very different applications of this new bound. Firstly, we obtain improved bounds on the classical computational resources necessary to simulate many-body dynamics with error tolerance 𝜀 for any finite time 𝑡: as 𝜀 becomes sufficiently small, only 𝜀 −O⁡(𝑡 𝑑−1 ) resources are needed. A protocol that likely saturates this bound is given. Secondly, we prove that disorder operators have volume-law suppression near the “solvable (Ising) point” in quantum phases with spontaneous symmetry breaking, which implies a new diagnostic for distinguishing many-body phases of quantum matter.

computational complexity↗

DOE OSTI2020

Linking electronic structure calculations to generalized stacking fault energies in multicomponent alloys

Abstract The generalized stacking fault energy is a key ingredient to mesoscale models of dislocations. Here we develop an approach to quantify the dependence of generalized stacking fault energies on the degree of chemical disorder in multicomponent alloys. We introduce the notion of a “configurationally-resolved planar fault” (CRPF) energy and extend the cluster expansion method from alloy theory to express the CRPF as a function of chemical occupation variables of sites surrounding the fault. We apply the approach to explore the composition and temperature dependence of the unstable stacking fault energy (USF) in binary Mo–Nb alloys. First-principles calculations are used to parameterize a formation energy and CRPF cluster expansion. Monte Carlo simulations show that the distribution of USF energies is significantly affected by chemical composition and temperature. The formalism is broadly applicable to arbitrary crystal structures and alloy chemistries and will enable the development of rigorous models for deformation mechanisms in high-entropy alloys.

Natarajan, Anirudh Raju (ORCID:0000000202529480)↗

DOE OSTI2021

Combination of cluster number counts and two-point correlations: validation on mock Dark Energy Survey

ABSTRACT We present a method of combining cluster abundances and large-scale two-point correlations, namely galaxy clustering, galaxy–cluster cross-correlations, cluster autocorrelations, and cluster lensing. This data vector yields comparable cosmological constraints to traditional analyses that rely on small-scale cluster lensing for mass calibration. We use cosmological survey simulations designed to resemble the Dark Energy Survey Year 1 (DES-Y1) data to validate the analytical covariance matrix and the parameter inferences. The posterior distribution from the analysis of simulations is statistically consistent with the absence of systematic biases detectable at the precision of the DES-Y1 experiment. We compare the χ2 values in simulations to their expectation and find no significant difference. The robustness of our results against a variety of systematic effects is verified using a simulated likelihood analysis of DES-Y1-like data vectors. This work presents the first-ever end-to-end validation of a cluster abundance cosmological analysis on galaxy catalogue level simulations.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

The cluster decomposition of the configurational energy of multicomponent alloys

Abstract The cluster expansion method (CEM) is a widely used lattice-based technique in the study of multicomponent alloys. Despite its prevalent use, a clear understanding of expansion terms is lacking. We present a modern mathematical formalism of the CEM and introduce thecluster decomposition—a unique and basis-independent decomposition for functions of the atomic configuration in a crystal. We identify the cluster decomposition as an invariant ANOVA decomposition; and demonstrate how functional analysis of variance and sensitivity analysis can be used to interpret interactions among species. Furthermore, we show how the mathematical structure of the cluster decomposition enables numerical evaluation that scales with the number of clusters and is independent of the number of species. Overall, our work enables rigorous interpretations of interactions among species, provides opportunities to explore parameter estimation beyond linear regression, introduces a numerical efficient implementation, and enables analysis of cluster expansions based on established mathematical and statistical principles.

Chemistry↗

DOE OSTI2023

Heterogeneous Graph Neural Network for identifying hadronically decayed tau leptons at the High Luminosity LHC

Here, we present a new algorithm that identifies reconstructed jets originating from hadronic decays of tau leptons against those from quarks or gluons. No tau lepton reconstruction algorithm is used. Instead, the algorithm represents jets as heterogeneous graphs with tracks and energy clusters as nodes and trains a Graph Neural Network to identify tau jets from other jets. Different attributed graph representations and different GNN architectures are explored. We propose to use differential track and energy cluster information as node features and a heterogeneous sequentially-biased encoding for the inputs to final graph-level classification.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Optimal Electrification Using Renewable Energies: Microgrid Installation Model with Combined Mixture k-Means Clustering Algorithm, Mixed Integer Linear Programming, and Onsset Method

Optimal planning and design of microgrids are priorities in the electrification of off-grid areas. Indeed, in one of the Sustainable Development Goals (SDG 7), the UN recommends universal access to electricity for all at the lowest cost. Several optimization methods with different strategies have been proposed in the literature as ways to achieve this goal. This paper proposes a microgrid installation and planning model based on a combination of several techniques. The programming language Python 3.10 was used in conjunction with machine learning techniques such as unsupervised learning based on K-means clustering and deterministic optimization methods based on mixed linear programming. These methods were complemented by the open-source spatial method for optimal electrification planning: onsset. Four levels of study were carried out. The first level consisted of simulating the model obtained with a cluster, which is considered based on the elbow and k-means clustering method as a case study. The second level involved sizing the microgrid with a capacity of 40 kW and optimizing all the resources available on site. The example of the different resources in the Togo case was considered. At the third level, the work consisted of proposing an optimal connection model for the microgrid based on voltage stability constraints and considering, above all, the capacity limit of the source substation. Finally, the fourth level involved a planning study of electrification strategies based mainly on microgrids according to the study scenario. The results of the first level of study enabled us to obtain an optimal location for the centroid of the cluster under consideration, according to the different load positions of this cluster. Then, the results of the second level of study were used to highlight the optimal resources obtained and proposed by the optimization model formulated based on the various technology costs, such as investment, maintenance, and operating costs, which were based on the technical limits of the various technologies. In these results, solar systems account for 80% of the maximum load considered, compared to 7.5% for wind systems and 12.5% for battery systems. Next, an optimal microgrid connection model was proposed based on the constraints of a voltage stability limit estimated to be 10% of the maximum voltage drop. The results obtained for the third level of study enabled us to present selective results for load nodes in relation to the source station node. Finally, the last results made it possible to plan electrification using different network technologies and systems in the short and long term. The case study of Togo was taken into account. The various results obtained from the different techniques provide the necessary leads for a feasibility study for optimal electrification of off-grid areas using microgrid systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2020

On Performance Prediction of Big Data Transfer in High-performance Networks

Big data generated by large-scale scientific and industrial applications need to be transferred between different geographical locations for remote storage, processing, and analysis. High-speed dedicated connections provisioned in High-performance Networks (HPNs) are increasingly utilized to carry out such big data transfer. HPN management highly relies on an important capability of performance (mainly throughput) prediction to reserve sufficient bandwidth and meanwhile avoid over-provisioning that may result in unnecessary resource waste. This capability is critical to improving the resource (mainly bandwidth) utilization of dedicated connections and meeting various user requests for data transfer. Conventional methods conduct performance prediction by fitting prior observed transfer history with predefined loss functions, without considering unobservable latent factors such as competing loads on end hosts. Such latent factors also have a significant impact on the application-level data transfer performance, which may result in an inaccurate prediction model. In this paper, we first investigate the impact of latent factors and propose a clustering-based method to eliminate their negative impact on performance prediction. We then develop a robust machine learning-based performance predictor by: i) incorporating the proposed latent factor elimination method into data preprocessing, and ii) adopting a customized domain guided loss function. Extensive experimental results show that our predictor achieves significantly higher prediction accuracy than several other state-of-the-art methods.

Liu, Wuji↗